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ABSTRACT 

The research methodology of a study to assess 
1980-1981 award accuracy of the^Basic Educational Oppor tun itiMlr ants 
(BEOG) , or Pell grants, is described. The study is the ,f iXst staq> of 
a three-stage quality control project. During the sprinaT of 1981 aV 
nationally representative sample of 305 public, private) and 
. proprietary institutions was visited. The financial add administrator 
at^each institution was interviewed and*askedto describe the 
instiJ£ution's BEOG processing procedure At each institution, data 
from a random sample of an average. of 14 financial a/d records were 
reviewed and transcribed. In all, data were collected from 4,500 BEOG, 
recipient records. These recipients and their parents Were 
interviewed, asked about they: general "experiences *in dealing with 
the application process, and asked to provide documents to verify the 
income and household -information on their application forms. In 
addition, data were collected' from the Internal Revenue Service, tax 
assessors, and financial institutions as additional "verification of • 
the information that the students in the survey placed on their 
applications.. Information is presented concerning the rationale and' 
specific procedures used to ^select a statistically representative 
sample of institutions and' students; the response rates for the 
survey of students and parents; institutional and student/parent 
field data collection; and procedures used to compile, edit, and 
convert the survey data to machine-readable tapes\sed for 
•statistical analysis. Attention is alsp directed to the methodology 
and procedures used, in the special analysis of BEOG application 
processor data entry error. (SW) 
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< CHAPTER 1 

INTRO DUCT I0N 

r 

9 PROJECT OVERVIEW ' 

' ^ ' ' 

The Office of Student Financial Assistance [OSFA] of the 

Department of Education has contracted with Advanced Technology, 

Inc. of McLean, Virginia", and ^stat, Inc. of Rockville, 

Maryland, to .conduct a three-year quaiity control study of the 

Basic Educational Opportunity Grant rBEOGl Progran. - The goal is 

to assess the accuracy and reliability of the BEOG progran and 

improve its administration. • m f 

The project is being conducted in three stages. During the 

first year of -the project — or St.age One — the objectives have been 

r . r - r 

to neasure program error rates, to determine the cause" of error, 

and to recommend changes, to reduce error. Specifically, the 

Stage One study has: f 

i * 

* • Determined program-wide discrepancy rates and probable 
causes attributable to (1)- institutions, (2) recipi- 
ents, ( 3 ) parents, and (4) application processors 

/ 

> • Identified "error prone" subpopulations- among recip- 
ients 

• Developed cost-benefit analyses for feasible corrective 
management activities to reduce -error rates for eyery 
area in which error rates are excessive 

In Stage Two, running from October 1981 to- December 1982, 

an ongoing Quality CQntrol System' will be designed and tested to 

continuously neasure -and analyze BEOG program pe/forriance. In 

Stage Thpee, running* from October 1982 to December 1983, the 

Quality Control System will be installed. 



ERIC \ 3 



REPORT OVERVIEW * * 

This is "a report on the methodology xised to accomplish the 
Staqe One objective^ just given. During the spring of 1931 ,a 

nationally representative sample of 305 public, private, apd 

w% ' * 

prdprietary institutions "was visited. The Financial Aid 'Adminis- 
* . * ' ** 

trator [FAA]> at each institution was interviewed and asked to 
describe the institution's BE0G- processing procedure. At each-. 

e w " 

institution, data from a random sample of an average of 14 finan- 
cial'aid records were reviewed and* transcribed * In all, data 
were collected. £rom 4,500 BEOG recip>i^.t records*. These 4,500 
BEOG recipifents--and their?\ parents — we^e interviewed, asked about 
their qeneral experiences./in* dealing wi-th the application pro- 
-ce&s, and asked c to provide documents to verify the income and % 
household information on their-application forms. In ^addition, 
data were collected from the IRS, tax assessors, and financial 
institutiorts as additional verification of the information that 
the students iiC^ke survey placed- on their applications. In sum, 
eight data sets from the. following sources were collected an* 

Analyzed to meet the Stage One objectives t 4 

» » 

• FAA interviews % - 

>% Financial aid records 

• Student interviews * 

m Parent interviews 

• Student Eligibility Reports TSERl 
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IRS records . * * 
* x 

Tax assessor records 



• Financial institution, records \ 

Chapter 2 of this report 'describes -the rationale and the 

specific procedures used to select a statistically representative 

sample of institutions and students. In addition, this chapter 

discusses the response rates for the survey of students and par- 

ents anfl analyzes the nonresponse to 'determine the possible 

nonres£onse bras in estimating program-wide e^rror amounts. 

Chapter 3 focuses, orjpthe^ins t itut ional and student/parent field 

.data collection. Included is a ^discussion .of thl field otjjarji- 

-zation, the quality control procedures, and the procedures used 

to^onduct t^e interviews . Chapter 4 "describes the procedures 

used to compile, edit, and convert the survey data to machine^ * 

readable tapes ^usecl* for statistical analysis. 'Chapter' 5 p<ovides 

a brief -discussion of the methodology and procedures used in <£he 

special analysis of BEOG Application processor data entry error 

<.The following summarizes, by chapter, the , general » features 

of 'the Stage One methodology. . 

Chapter 2 — Sample Selection and Sampling Error 

• Tn designing* the sample, the goal was to provide as 
precise estimates as possible. of the* universe under 
study and at the same time: , 

Limit the' amount of field traveJL 

Control the number of student* and separate institu- 
tions selected 



variety *o^fcyp< 



- Ensure that a variety ^o^fcypes of institutions were 
represented in the sarftple 



To accomplish this, institutions and studerfjts were 
selected in two stages: (1) stratification and random/ 
selection of 305 institutions, and (.2) stratification 
and random selection of. approximately 4,500 BEOG 
recipients from these institutions. Both the institu- 
tion and student ^sample sizes were large enough to 
ensure the required statistical precision. 

Institutions were stratified into four groups: 

- 'institutions with over 5,000 BEOG eligible students 

Institutions using the Regular Disbursement System * 
RDS] and located in the ^25 largest Standard 
Metropolitan Statistical Areas . [SMSAs ] 

•** * ' * A 

Institutions using RDS and located outside the 25 

. largest SMSAs m 

Institutions using' the Alternate Disbursement System 
[ADS] A * 

All ^institutions in the first group were included in 
"the 'institutional sample , -Institutions within ^ the 
other three were 'clustered geographically- *to limit ^ • 
travel costs and further stratified by ^institui^ipnal ^ 
typ^and control ( i .e .*, public, private, and pro- 
prietary), 'institutions were selected from' each sub- 
stratum with probabilities proportional to siz£. 

Recipients from ee^ch of the 305 selected institutions 
were strati f red by grant amount to provide bettef: 
estimate of the total absolute dollar error*. It was 
necessarily. sample "validated 11 recipients at a higher 
rate than unvalidatecj recipients in order -to provide 
more accurate estimates of this analytically- important 
" subgroup. ^ ' % 

Over 90 percent* bf all students and parents in the sam- 
ple ^responded with an interview. This composite re- 
sponse rate- 'can be broken down for dependent and inde-- 
pendent students, and parents"*^ follows: 

- Dependents: ' students: * 94.6% 
• * • 4 parents : 93.6% 

Independents: students: 8*7.4% 

^parents : ^ *61.0% 



t 



• The analysis thus far shows that nonresponse in the " 
survey will not siqnif icantly 'undermine .the study's 
overall findings. Preliminary analysis shows that 

j nonrespondents do not differ signif icantlyS^rom 

respondents in terms of a variety of characteristics. 

Chapter 3 — bata .Collection and Quality Control 

• Responsibility for data collection was' divided between 
Westat and Advanced Technology. Westat conducted 
nationwide interviews with students and parents and 
collected secondary data from, the IRS, tax assessors, 
and financial institutions,. 'Westat 1 s data collection* 
occurred from February to April 1981. .Advanced 
Technology collected institutional data. site visits 
to, institutions .occurred from March to May 1981. 

• • Data collection questionnaires were developed from * 

those used in the 1978-79 Quality Control study. In 
order to ensure their effectiveness, they were pilot) 
tested in the Washington, D.C., area among a smalL*/ 
representative sample of . students, parents, and 
institutions . 

4 

• Westat used -its nationwide network to recruit approx- 
imately 200 interviewers to conduct the student and 
parent interviews. Nearly all had previous survey 
research interviewing experience; most had worked with* 
Westat on previous studies. All student/parent inter- 
viewers underwent rigorous training in interviewing 
techniques and field procedure. t 

. * Advanced Technology ' recruited 13 individuals to collect 
the institutional data. All had considerable student 
financial aid experience; 'several had advanced degrees. 
It was felt that well-educated persons with recent work 
experience in financial kid' offices -.were needed to 
conduct the specialized interviews v/ith Financial Aid 
Administrators. The, institutional interviewers were 
provided with one week off intensive training in the use 
of both the institutional questionnaire and student 
record* abstract form. 

• Several procedures were instituted to guarantee the ■ 
timeliness and integrity of the data collection. stu- > 
dent/parent interviewers were managed by seven full- 

A / time in-field supervisors. These regional supervisors*, . 
reported -each week to the home, office by telephone and 
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by using a compj*ter assisted management systjen. The 
institutional interview's were supervised directly by 
the hone office and were required to - report each. week 
by telephone ♦ For both survey operations, updates to 
procedures were communicated tq interviewers by fre- 
quent field memoranda. The -followingv were procedures 
used* to ensure the reliability of the data: " , 

Each*questionnaire was edited f6r omissions, 
ambiguities, or misplaced codes by the inter- 
viewer, by the field supervisor/ and finally by the 
coding staff in the home office. 

A random sample of approximately 10 percent of all 
completed cases was validated. Students, parents, 
and Financial Aid Administrators were telephoned, 
asked to comment on the conduct of the interviewer, 
and re-asked two or three key questions.^ 

All interviewers were monitored in the field by 
senior staff at least once during the data 
' , collect Jbn . ' 

Chapter 4 — .Data Processdng ^ 

• All survey questionnaires and secondary data were 

compiled / edited, and converted to nachine-re.adable 
tapes using standard ADP procedures. Standard 
procedures included: 

^ - Preliminary manual edit to- check for completeness 
and consistency 
* * 
Coding for data entry 

Another manual edit for coding and interviewer 
errors 

Key entry and key verification 

Machine edit for omissions, consistency, and va'iid x 
data ranges 

Frequency distributipns of each variable to check 
for data errors not detected by manual and machine 
edits 
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, • When significant errors or 'omissions were discovered 

' during 'the manuai or machine edits, the respondent Or 

. vVr ' interviewer was telephoned to retrieve the correct 
* data t . , , . ' 

• Once edited and entered ontQ tape, the clean data files 
were reformatted and o merged into a master file for 
access by SAS, a statistical software package. 

Chapter. 5 — Application -Processor Data Entry Error Analysis 



K ra-ndom subsample of 1,250 BEOG applications was drawn 
from our sample to analyze the error rate associated 
with data- entry. All sub'sampled applications origin- 
ated from the Multiple Data Entry [MDE] prpcessors. 
The. disposition of the subsample, by proce^or, - 
follows : 

500 - College Scholarship Service [CSS] 

500 - American College Testing [ACT] Program 

250 - Pennsylvania Higher Education Assistance 
Agency [PHEAA] 

The original applicati6n forjns were visually compared 
' with the data contained on the 1980-81 Central 
Processor's History/Correction File. Discrepancies 
were tabulated manually. 

t 
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CHAPTER 2 
SAMPLE SELECTION AND SAMPLING ERROR 

SAMPLE DESIGN AND SAMPLING ^PROCEDURES 

Des.iqn Objectives 
— j 

The primary objective of the sample design was the selection 
of a probability sample of students' enrolled at educational 
institutions involved in the 19R0-81 BEOG progr'am. The statis- 
tically representative sample was used to document , compute, and 
analyze the program-wide error rate* in progtam determinations 
including eligibility, benefit \ anoufi^:, and payments • To do this, 
data were obtain-ed from students, their parents^, and educational 
institutions. f v . - 

Itt terms of survey implementation, the* sample design had the 
following objectives: 

• Limit the amount and cost, of field travel 



• Control the number of students and separate institu- 
tions Selected 

• Ensure the representation of a variety of institution 
types 

• Provide as precise estimates as possible 

Tne procedures described in the^ following^ which empT^S 
selection of units with probabilities proportionate to size, 
yielded a sample meeting these objectives* Development of esti- 
mated measures ojj^ size is discussed in the following, paraqraphs • 

An important goal in sample design is to organize available 

information for *:awing, the sample so as to*reduce the variabil-^ 

/ 

lty of the characteristics of possible samples or to increase 
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the precision of estimates. Ideally, the sample design would 
.result in a set of possible samples each of which closely resem- 
bles the universe under study, while assuring' all units a known 
chance of selection. 
Measure of Size 

In an earlier survey of BEOG recipients, the number of 
students eligible for a grant "at an institution was *used to - 
assign a selection probability for the institution. The es£i- 
riated figures were used to obtain a "measure of size" [MOS] for 
the first-stage selection based on probabilities proportionate to 
'size. When the MOS is * not well correlated with the statistics ^ 
being estimated, one of t^o conditions will result: either the 
number of students in the sample will be subject to greater 
variations than desired or great disparities in selection proba- 
bilities for students will occur. When the Matter happens, sub- 
stantial inefficiencies or large sampling errors can result. 

The institution-reported number of eligibles was obtained 
from Department of Education data files. It is not surprising 
th$t these figures rarely matched the actual number of recipients 
at the time the institution was contacted. Frequently, this .was 



due to the fact that 'eligible students do not always apply for 
grants each semester. Thus, the number of eligibiles generally 
overstated the. size of the universe at the time of sampling. In 
the selectio.n scheme described in the^llowing, the estimated 
number of eligibles was used to improve the design in a manner 
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somewhat different from that of previous studies. TJiis method is 

» *» 
less affected by the accuracy of the .estimates: 

Size of institution was involved at two^points in the 
design.-" First, the number of eligibles, indicated on the OE data 
tape, was ujsed in the formation of geographic clusters of insti- 
tutions. These clusters were" v constructed so as to eontain 
roughly equal numbers of eligibles. For thLs purpose, ' accuracy 
of the size measure was relatively less critical than in the pre- 
vious study. 

-After clusters were -selected, the schools in th^ clusters 
were" contacted by telephone. This, fairly inexpensive ef fort . was 
aiftied^ at obtaining more precise and accurate estimates of the 
actual number of recipients dXf each institution,. With such mea- 
sures, a subsam^le of insti tiytiops was designated which allowed 
for greater control of^&tiy££nt sample size ahd of sampling 
errors. 

Desired Sample Seizes 

0n$ study^bjec/ive was to ensure that the confidence inter- 
val for national estimates of the case and dollar error for , 
program-wid$, student, and school groups not exceed plus or minus 
'5 percent with 95 percent confidence. Thus, ~the coefficient of 
variation (standard deviation divided by the mean) should be less 
than .025 for the overall estimate of the case error rates and 
dollar error amounts.' Below we 'discuss the implication of this 
requirement on the number of ^udents required in the; sample. 



During the previous study of 1978-79* recipients, it was 
estimated that approximately 84 percent of all institutions had 
an average net dollar error of practically zero (less than $2) 
To estimate this characteristic with the required precision, a 
sijnple random sample of 305 institutions would be required. That 
is if the square of the coefficient o'f variation is expressed as: 



V 2 = (1-P) 1 

P • P ' n 



This formula can be solved for desired sample size: 



(1-P) 1 
n = - 51 . • — 

P v 2 
P 

If p, the proportion of interest, equals .84, then the number/Of 
schools required to restrict the relative variance to (.-025) 2 is: 



n = il 6 . = 

(. 84) (. 00625) JUD 



This characteristic is just one of many statistics of interest; 
therefore, use of this sample size will not produce equally 
accurate estimates for other characteristics. 

The most important characteristics to be estimated pertained 
to students, not to institutions. Statements about the percentage 
of students with particular characteristics are more germane to 
the study than observations- about institutions * Below we discuss * 
the number of Students desired in the sample. 
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An important student characteristic is that of absolute 
dollar error. One estimate of the dollar error can be ^modeled 
simply as the product of two rafiQom variables: 
x= Re 

whe re 

x = average absolute dollar value of errors 

f 

P - the probability of making an error 

» 

e = the average of the/absolute dollar value of the 
error given that an error was made 

The coefficient of variation of such an estimate based on a 

simple random sample of n units is expressed as: 



nP 



where « ? 

Q = .1 - P \ 

= the 'coefficient of variation of the amount of 
error for those cases which are in error 

n = the number of, units (schooLs or students) 

This formula can be rearranged to develop an^expression for 

desired sample size: 



V 2 + Q 
n -'-2 



P(V^) 



Determination of the number »of students required in tKe sample to 
yield 'the estimated coefficient of variation requires estimates - 



of V* and P. Based upon our previous study, V* = . 935 and P. = 
.431. Using these values, a sample. of 5,600 students would be 
needed: 



n = .935 + (1-. 431) = 5583 

i* (.431) ('.025)* ■ * • 

We felt that the value of V* in this study woui^be consider 
ably below.. 935 for two reasons: (1) The method of selecting 
clusters differed, from the prior study in a way that reduced 
homogeneity within the clusters. This resulted in smaller 
between-cluster variances and in reduced design effects. Sample 
clusters would be created by intentionally combining schools of 
different sizes and hopefully reducing the rate of homogeneity 
(ROH) . (2) The avetage cluster size would be reduced. By 
forming smaller clusters of roughly equal size and by sampling 
more of these clusters; as "discussed below, any undesirable 
Effects of clustering would fcfe reduced still further*. x 
'The design effect is given by: 

deff = 1 + (ROH) (B-l) • •„ , 

" v With the rate of homogeneity (ROH) and average cluster size 
(B) both decreasing, we^expeoted a considerable reduction in V*. 
From our experience, it appeared that V^shou-ld be lowered to 
the' 0.4 to 0.5 ra'nge. - At 0.5 the required sample size was about 
4,000 students: 



n,= .5 + (lr.,431) : 3^ 68 



./ . . (.431) (.025)' - 



\ 

3968 • 

2 '_. 



Institutional .S^r^^^am^ 

The sample, ^^^^^qd. fjor selection pf -institutions is the 
Institutional Master^ili^^^I^mhintained by the Office of Stu- 
dent Financial 'Assi^ta^ace^v^ms file is created each year from 
the institutions approved? ^p^^artfeipa^ion^ and ^cfeta are added 



ag, the grant program is inpleraented 'and qaarterly reports are 



received. The 1979-80 f iie^.was^ed 'The file identified the 
most up-to-date list of participating institutions and the number 
and value of grants awarded in the previous academic year. 

For sampling purpos.es, itVa^ desirable to have a comprehen- 
sive listing of schools^ participating in *the BEOG program and to 
have, for each of thes^ schools, the number of grants given for 
the academic year 1980-81 'at the time of the survey- This latter 
figure served ideally as a measure bf size in drawing a sample 
schools,. It is believed that the participating schools will be 
adequately identified by the 1979-8CK I,MF, as described above • 
The number of grants given in the previous year was used as proxy 
for the 1980-81 grants where such information was available on 
the -IMF . S ' ° 

Use of previous year grants resulted in several problems. 
First , the i n stitutional sample need ed to sel ppI- branch ^npn^c 



of nulticampus institutions independently to avoid excess travel 
costs. However, the num ber of grants awarded to. a .nulticampus 
institution is* usually included in the central campus record;' 
ttlanks are recorded for k number of grants to the branch campuses. 

A 
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^Therefore, it was necessary to allocate the total" number o| 

grants anong the centra^, institution and -its branches,' 

If enrollments for the main campus and the branches were 

available # these enrollment data were used to allocate the total 

number of grants to th^j2*anches , If enrollment data were not 

available, the humber of grants was distributed using other 

assumptions. As in the previous survey, we assumed that the main 

campus had 'three times the number of grants of any branch and 

that "all branches had an equal number of grants, 

** 

Recipient Sample Frame - ' 

The recipient sample frame was institution records of grant 
.recipients as of the early part of 1980-81 academic year. This 
frame was expected to contain virtually all 1980-81 grant year 
recipients. However, certain types of students were dmitted such 
as late grant applicants, students whose applications- have not 
been approved, students enrolling for second term, etc. 
Sample Selection Procedures ' ' * , 

The sampling procedures involved two, phases: (1) selection 
of a s sample of institutionsXand ( 2T^^teat ion of a sampling of 
students at the selected institutions. * * 

Selection of Institutions * * 

The firs\ step in selection of tfe institution sample was 

♦ 

the slxrting of schools from the . master file into four groups: 

• Institutions with over 5,000 eligibles' 

• Institutions using Regular Disbursement System [RDS] 
lodSted in thd 25. largest SMSAs 



• Institutions using Regular Disbursenent System [RDS] 
• v not located in* the 25 largest SMSAs 

• Institutions u£ing Alternative Delivery System [ADS] or 
institutions for which there were no available eligi- 
bility counts on the master file 

The sample size was allocated to these four groups accord- 

ing .to the number o£ eligibles. For the fourth group, eligibi- 
l- 
ity counts were estimated using data from the earlier BEOGs 

study. All institutions -in the first group were included in the 

institution sample. Institutions' were selected from the other 

three groups as described in the following paragraphs. 

Institutions Using RDS in the 25 Largest SMSAs 

Sample selection involved five steps for this group: 

1. Select a systematic sample of about 150 institutions 
from a list of all schools ordered by SMSA, type of 
control, type of school, and number of eligibles. 
Schools with large numbers of eligibiles, i.e., wh£re 
the number of eligibiles exceeded the sampling inter- 
s' ^ ^ al/ were included with certainty at this stage. 

2. Contact the schools selected above, and obtain an 
improved measure of size i.e., number* o^f recipients. 
This new measure of size will be used in the final 
probability proportional to size selection stage. 

3. Form substrata using type of control, type of institu- 
tion, and size (eligibility cbunts). 

' 4. Allocate sample to substratum -according to new measure 
of size (recipients) . 

5. Select desired sample from 'each substratum with proba- 

' — bi-1 i ty- pr apart ionaJ to. jjsjl&ej. 



Institutions Using RDS Not Located in 25 Largest SMSAs 



• Institutions in this group were selected using" an eight-step 
procedure. ^Geographic clustering was utilized for this group in 

order to control » interviewing costs. 

»« * *^ 

1. Sort institutions according to geographical location 
using Westat's Zip Recode., 

v "2 < Form geographic clusters of nearly equal size. Nearly 
equal size means that the smallest cluster in terms of 
the number of eligibles cannot* be less than one half 
the average size nor greater than twice the-average 
£ size. Clusters- .formed by grouping multiple institu- 

tions most often yielded a work load of 20 to 29 cases; 
single institution clusters most often produced 30 tq, 
39 cases. 40 

3. Combine the above clusters , into 40 primary strata by 
counting off one-fortieth of the clusters. 

4. Select two clusters per stratum using- equal probabili- 
ties without replacement. 

* S. Contact schools in the 80 clusters and secure anew 

measure of size, number of recipients ♦ These measures 
were collected using* telephone calls. 

6. Stratify institutions according to type of control, 
type of institution, and size (number of eligibles). 

7. Allocate sample to substrata according to new" measure 
of size, number of, recipient^. *• . 4 

Select institutions from each substratum with probabil- 
ities proportional to size. _ '4 

Institutions Using ADS or Institutions Where Number of Eli— 
g ibl e s Is Not Known mmm 

Institutions in this group were selected using a two-step 



procedure . 



1. Sort by whether institutions were inside or outside the 
25 larges/t SMSAs, type of control, and type of institu- 
tions* - . 



2. Select institutions from the entire list using' system- 
atic sampling with equal probabilities. * 

The results of these procedures -are summarized in the^fQl- 

lowing three tables. /The first twp^tables cover the certainty ' 

"iff* * 

groups (rows with size of institutiohs over 5 , 000 ) 5 ao|^he two 
RDS groups. fI? ^%i^ t table includes ^DS schools and Tchools for 



which r weT had no ^.iz.^ "estimates . 

Selection of St^j^nts from .Samplg<fe*Ins titutions • " J 
Lists of students receiving- Basic Grants were obtained from 
^the sampled institutions during a^eriod oj? eight weeks. If' the 
lists were not received in 'the mail on a timely basis, on-side 
visits were., made to^ select a sample. Students were stratified by 
dollar grant amenta, to provide better estimates of the total 

absolute do ££ax , errors ; . ^ 

, ' \ S " \ • • * . - * 

. v ^.Imosder to^as'sure that at least 1,000 students identified 

by ,lhe lustrations as "validated" students were ihc'luried in the 
sample, we asked each institution to identify the total nunfber of 
students to be validated from their institution. Then, depending 
on the number of total students to be validated, t we separated- the v 
two lists and selected proportionately from the two lists of 
recipients. This provided, us with at least* T, 000 "validated 11 
students.' t . 

. • -* 
Detailed Procedure . 



1. To ensure- accurate probabilities in student selection/*" 
separate lists sorted by, grant Amouijfc were compiled, for each * 
-validation status, smaller "institutions all recipients' * 
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INSTITUTIONAL UNIVERSE AND SAMPLE: 
ADS STRATA 



names and addresses were written on the appropriate sheet, thus 
effectively stratifying students by grant amount. For larger 
institutions, particularly when sampling on-site, a sanple of l""^ 
in k students was selected, where k is chosen^ to yield no more 

7 v » 

than 300 names. Only the sampled .names were stratified by vali- 
dation status and sorted by amount. 

2. After the listing sheets had been completed, a system- 
atic sample was drawn from each list.. It was necessary to sample 
validated students at a higher rate. The sampling fractions were 
established so' that the final samples would be self-weighting 
within the two groups, validated and unvalidated. It was neces- 
sary to sample validated students at a higher rate to achieve the 
desired sample size. 

3. Based on the desired 4,000 completed student instru- 
ments, 1,000 of whom were validated from 305 participating insti- 
tutions, and an anticipated completion rate of .87 (this was the 
completion rate for the 1978-79 study), an average of 15 recipi- 
ents per institution" were selected. ' • 

* ' / 

While the twb samples are se If -weight ing , the sample weights. 

for unvalidated students are 6 times (635) as high as the sample 

weights for validated students (105.8).' Therefore, estimates of 

overall aggregates, averages, and proportions have . to be weighted 

estimates. 
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ESTIMATION OF SAMPLING ERROR 



Student Characteristics 



Every estimate of a total, &, can be expressed 4s the sum of 
foyr components, as follows: J 



r\ * ^ 

x = x 1 + x 2 + x 3 +, x 4 1 



(1) 



where X^ is the estimate from the certainty schools, & 2 is the 
estimate from the ADS schools, X 3 is ttfe' estimate from the,RDS 
schools in the 25 largest SMSA's; and X 4 is the estimate from the 
remaining schools. Since the samples were drawn independently in 
each stratum, the variance of X is the sum of the variances of 
the four components. That is, 



Var X = Var X 1 + Var £ 2 + Var & 3 + Var £ 4< 



(2) 



The varia'nce of a ratio R = Y/X may be approximated as 



Var R = 



Y 

A 

X 



Var X 



Var Y 

~ 2 
• Y 



2Cov (X, Y) 
XY 



where^ 

.Cov(X,.Y) ■ Cov^Xj,^) + Cov(X 2 ,Y 2 ) + Cov(X 3 ,Y 3 ) + Cov(X 4 ,Y 4 ) 
&We examine these components separately. 
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In the certainty schools, the selection of students provides 
the only source of sampling error in pur estimates. Withih each 
school students were separated into two lists, validated and 
unvalidated, sorted by grant amount, and then each list was 
systematically sampled. An estimate of Var X^ is then 

n n v-l I 

Var X, = Z Z ^-r— — ^ £ (Vix. - w. ,x. 1 ) 2 

1 s ves 2 ( n v - 1) i=l ii - l+l l+l ; • ' 

where , 

i denotes the individual case in its order of selection; 
w£ denotes the case weight; 

v specifies whether the Case is validated or unvalidated; 
n v is the number of validated [oh unvalidated) students 

responding in a particular school; and 
s, denotes the school. ' 

Similarly, CovfX^Yj) can be computed as 

. s vcs .2(n^ I 1) (w i x i " w i+l*i +l > (W i y i " W i+lW 

Within a few schools, validated and unvalidated respondents were 
pooled to fiacilit^te computation: * * 

The sample of ADS schools was selected from a sorted list 
using systematic sampling with equal probability. 



eric 
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An est imate * of Var is then: 
* V " X 2 "2(n- S - 1) ,X S " "s + l> 



where ' 

s * indexes a school in its order'bf selection; 

n g equals the number of ADS schools responding; 

x 1 equals Z w.x. , the total for school s 
s ies 1 1 

i denotes the individual case; and 

. denotes the case weight. 



/\ /s. 



Cov(X 2 ,Y 2 ) is computed as 



n * n s -1 



2(n s - 1) s =i 

/ 

The RDS schools within the 25 largest SMSA'S were ordered by 
SMSA, control type, schQol type< and number of eligibles prepara- 
tory to selecting a preliminary sample of schools systematically 
with probability proportional to the number of eligibles. Al- 
though the sample was drawn with a single random start, for the 
purpose of variance estimation we assume that we have independent 
strata defined by SMSA and estimate Var & 3 a s. follows: 



.?0 
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n SMSA _ 0 

Var = I n I (x' - x ' ) A 

3 SMSA SMSA se SMSA S SMSA 



where „ 

denotes a school; 

n SMSA e( 3 uals th ^' number of schools responding from a 
particular_£MSA; 

x' eqxialjSZ w.x.«, the total for school s; 
& ies 1 1 ' 

; i 

"SMSA seSMSA " s 
total within an SMSA; 
i denotes the individual case; and 

'W. denotes the case weight. - 



SMSA e< 3 uals ft" 2 , x' , the average school level 



Cov (X3Y-) is computed as 



n SMSA _ ,. \ - f • 

n SMSA -1 seSMSA S " X SMSA° (Y S " y SMSA } 



SMSA SMSA 

In two cases there was only a single school responding in an' 
SMSA. Where this happened, pooled data from the SMSA adjacent 
in our sorted list to facilitate the calculations. 

Tfie RDS schools outside the 25 largest SMSA' s were- ordered 

and grouped into cluster-s of roughly equal. size. Clusters' were 

grouped into, 40 strata. The first stage-of sample / selection ' 

chose two clusters per stratum with equal probability. An 

expression for the variance estimate Var X is 

. 4 



^. ~» T *■ » " - 




4 










■ 


1* ~* * - ~v 










, -where.^ . * • > , * " 
^denotes tire cluster ; - . ■ 






i " 


"* h ^ ^ denotes^ the stratum;'"' 

iW ' ; edual*s^th"e number, of- clusters responding from 








" ^'-v^to^tiinr. fc;^ ^ . , * % — ^ 








'X 1 equal's 1 Z w.x. the total fcgu^cluster c; . 
c * iec 1 1 ^ j 

■ . ' • \ " 

x' equals — Z x 1 the average cluster total in 
h n h ceh c * * 
stratum h; - ■* - - - 

i denotes the individual case; and 4 








w. denotes the case weight, 
i 

,» - - v 


- 












Similarly, Cov U A ,Y.) is computed as 
«^ 4 * 








40 n. 

1 h . 2 (x' - x') (y' - yJ) . 
h=l n h -l ceh c h c h 
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Estimated Sampling Errors • 

Icu-this section we present estimated sampling errors 

developed using the methods described in the previous section. 

* * » "* 

For each statistic we* present the estimate itself, the standard 

error of the estimate, and the coefficient of variation (standard 

error of the estimates divided by the 'estimate). Estimates pre- 

sented in these* tables will not agree strictly with estimates 

presented in the- text o.f Volume 1 for > two reasons-.* First, the 

estimates presented here were developed using software procedures 

different from the software utilized for Volume 1 estimates. 

e 

These differences in computer rounding and truncation can be 
expected to account for large portions of the small differences. 
Secoild, development o£ the estimates presented here is based on a 
slightly different methodology concerning assumptions about 
missing or incomplete^data. * - 

% Figure 2-4 presents the Standard 'errors of estimated aggre- 
gate payment errors. The coefficients of variation for student 
and case error are always less than 11 percent, and the coeffi- 
cient for total absolute 'case error is only 6 percent. Estimated 
standard errors for number of cases with payment or eligibility 
errrors are presented In Figure. 2-5.^ 'The standard erjrors for 
estimated ^numbers of payment 'errors are* generally below 10 per- 
cent. Occurrence of eligibility erorrs, except AEP and FAT, is 
extremely rare. For the course length requirement we uncovered 




ERLC 



2-21 2Z * 



only one error in our entire sample. Therefore, the coefficients 
of variation will appear to be high.' However, the standard 
errors are low enough so that reasonably tight confidence 
intervals exist. 

Standard errors for average payment, errors are presented in 
Fi^«?e 1-6. These standard errors are generally $20 "or less; 
thus, 95 percent confidence intervals would generally have a 
width .of $80 of less. 

The remaining tables provide sampling errors for the various 

estimates for the following categories: 

Type of Control 
Type of Institution 
Grant Type 
Validation Status 

Figures 2-7 to 2-18 aire for ther various aggregate payment 
errors; Figure 2-19 to 2-24 are for numbers of cases with payment 
errors; Figures 2-25 to -2-S6 contain sampling errors for average 
payment errors; and finally, Figures 2-37 to 2-45 are for the 
estimateiPnumber of cases with eligibility errors. 

"Comparing esFimate^ the 
earlier 1978-79 study and the current study reveals that tor many 
of the overall estimates the current study almost always yielded* 
lower coefficients of variation. However, the differences in 
survey instrumentation, field work procedure^,, grant levels, 
etc./ between the two studies imply that any comparisons such as 
these must be done with extreme caution. 



i 

ft 

Statistic 


Estimate 
($ millions) 


Standard 
Error of 
Estimate 


1 Coefficient 
of Variation 


/ ■ 

OVERPAYMENTS 


». 




V 


Institution 

Student 

Case 

UNDERPAYMENTS 


264:05 
263.44 
511.73 


30. 06 
18. 96 
37. 00 

t 


0.11 
0. 07 
0.07 


Institution > 

Student 

Case « 

NET PAYMENT ERROR - 


93.65 
46.75 
125.77 


12.85 
4. 75 
13.31 


0.14 
0.10 
0.11 


Institution 

Student 

Case 

ABSOLUTE PAYMENT ERR.OR - 


170.40 
216.69 
385.96 


33.00 
17.09 
37.50 


0.19 
0.08 
0.10 

f 


Institution 

Student 

Case 


357.71 
310.19 
637.51 


32. 38 
21. 73 
41.07 


0.09 
0.07 
0.06 



FIGURE 2-4 



ESTIMATED SAMPLING ERRORS: 
SELECTED AGGREGATE DOLLAR ESTIMATES 
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Statistic 


Est unate 
(Thousands) 


Standard 
Estimate 


Pnof f i f"ipn t 

of Variation" 


Institution Overpayment 
Institution Underpayment 
Student Overpayment 
Student Underpayment 
Case Overpayments 
Case Underpayments 


597.60 
373.27 
665. 43 
203.08 
1,141.84 
^1.04 


55.25 
43.36 

4i. sa 

17.47 
66.85 
45.12 


0.09 
0.12 
0.06 
0.09- 
0.06 
0.. 09 . 


ELIGIBILITY ERRORS 








Academic Progress 
Enrollment Status 
Course Length ; 
Degree Program 
Possesses B. A. 
Missing AEP 
Missing FAT ^ 
Loan Default^ 

Citizenship ; 

? 


-28.58 
2.27 
.68 
• 1.36 
4'. 08 
92.90 
102.28 
.91 
1.36 


8.56 
1.19 
.68 
.96 
1.69 
25.11 
15.04 
.70 
.96 


0.30 
0.52 
1.00 

a>7i 

0.41 
0.27 
0.15 
0.77 
' 0.71 



FIGURE 2-5 



ESTIMATED SAMPLING ERRORS: 
SELECTED ESTIMATES OF CASES WITH ERRORS 



'j r 
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* 






- 


j 

Statistic 


Estimate 
* . (Dollars ) . 


7— 

' Standard 
Error of 
Estiwate 


Coefficient 
of Variation 


AVEaAGE overpayment 




• 




Institution 
/Student 
Case 


442 
396 
448 


34 
17 
20 


0.08 
0.O4 
0..05 


AVERAGE UNDERPAYMENT 








Institution 

Student 

Case 

m 

/WtiAAvjti NE1 EKKUK ■ 


251 
230 
256 


19' 
13 

> 15 


'0.08 
0.06 
4 0.06 

- 


Institution 

Student 

Case 


74 
93 

.. 167 


' 14 
7. 
16 


, 0.19 - 
1 0.08 
0.10 


AVERAGE ABSOLUTE ERROR 

Institution 
, Student 
Case 


•- , 156 

; 133 1 

, 276 

a 


13 
8 
14 


0. 08 
0.06, 
0.05£ 




• * 
* 

FIGURE 2-6 , 







ESTIMATED SAMPLING ERRORS:' 
SELECTED AVERAGE DOLLAR ESTIMATES' 



* 


•9 

Category 


Estimate 
($ millions) 


Standard 
Error 

* 


Coefficient 

of Variation 
* 




TYPE OF CONTROL 


• 




■ 


• 

t mm ' m 


Public 
Private 
/'•Proprietary 


169.93 
55. §8/ 
38*44 

t » 


27. 40 
12.77 
12.07 


0.16 
0.23 
0.31 

> 




TYPE OF INSTITUTION 


i 








Four-Year 

Two-Year 

Other 

GRANT TYPE 


166.54 
72.67 
24.84 


23.98 
17.42 
9.89 


0.14 
• 0.24 
0.40' 


> 


Independent 
• Dependent 

t 

VALIDATION STATUS 


, 124.75 
139.30 


16.70 
17.37 

. 0. 


0.13 
0.12 




Non-Validated 
Validated 


251.82 
12.23 


28. 53 
2.12 


0.11 
0.17 - 




ALL CATEGORIES COMBINED 


264.05 


30.06 


0.11 



FIGURE 2-7 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: , 
- ' INSTITUTION OVERPAYMENT 
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Category 


Estimate 
($ millions) 


4 

Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








Public 


161.05 


16.34 


0.10 


Private ^ 


71.03 


11.71 


0.16 


Proprietary 


31.35 


8.08 


0.26 


TYPE OF INSTITUTION 




t 




Four-Year 


190.09 


18.07 


0.10 


Two-Year 


'53.56 


7.79 


0.15 


Other 


19.79 

- 


4.94 


0.25 


GRANT TYPE 




* 




Independent 


99.94 


11.86 


0.12 


Dependent 


, 16-3.50 


12.98 

- i 


0.08 


VALIDATION STATUS 


• 






Non-Validated 


252. 13 


18.70 


0.07 


Validated 


11.31 


1.15 


0.10 










ALL CATEGORIES COMBINED 


263.44 


18.96 


0.07 



* FIGURE 2-8 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
STUDENT OVERPAYMENT 



Category I 


Estimate 
($ millions) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 




a 




Public 

Private 

Proprietary 


124.18 
64. 51 


7 5 1 ^ 
j j i i j 

20.46 
16.19 


0 11 

^ U . XX 

0.16 
0.25 


TYPE OF INSTITUTION ° 






at 


Four-Year 
Two -Year 
Other 


jjU . ZD 

121.05 
40.42 


84 
21. 81 
12.70 


0 . 09 
0.18 
0.31 


GRAN.T TYPE 








Independent 
Dependent 


' 217.69 
294.04. 


22. 00 . 
22.3 4 ■ 


0.10 

or. 08 


VALIDATION STATUS ' v 








Non-Validated 
Validated 


488.61 
23.12 


3b. 7U 
2.41 


U . U / 

0.10 


ALL CATEGORIES COMBINED 


511.73 


37.00 


0.07 



FIGURE 2-9 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

CASE OVERPAYMENT 



I 



4-o 



2-28 



Category 


Estimate 
( $ millions) 


Standard 
Error 


, Coefficient 
of Variation 


TYPE OF CONTROL 




- 




Public 


61.02 


11.71 


0.19 


Private 


5.72 


. 1.46 


0 . 26 


Proprietary 


26. 90 


6.69 


0. 25 


TYPE OF INSTITUTION 








Four-Year 


32.46 


8. 26 


0.25 


Two— Y6ar 


4j0 . 33 


8 . 84 


0. 22 


Other 


20.86 


6.61 


0.32 ». 


GRANT TYPE 








Independent e 


49. 05 


9. 07 


0. 18 


Dependent 


44.60 


7.10 


0.16 


VALIDATION STATUS 








Non-Validated 


90. 18 


12.73 


0.14 


/Validated 

/ " ' 


3.46 


0. 56 


. ,0.16 


ALL 'CATEGORIES COMBINED 


93. 65 


12.85 


0.14 



FIGURE 2-10 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES; 
" . INSTITUTION UNDERPAYMENT 
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Category 


Estimate 
($ millions) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








Public 

Private 

Proprietary 


29.26 
13.42 
4.06 


•3.90 
2.67 
2.17" 


, 0.13 
0.20 

0.53 ' 


TYPE OF INSTITUTION 








Four-Year 
1 Two -Year 
Other 


34 77 
8.60 
' 3. 37 


4.21 ' 

1.63 

2.13 


0.12 
? 0.19 
0^63 


GRANT TYPE 






- 


Independent 
, * Dependent 


9.17 - 
' 37.58 


1.93 
4.31 


0.21 
* 0.11 


VALIDATION STATUS 








Non-Vali*dated 
Validated 

• 


% 

4 

43 . 94 
2.81 

\ 


ft • 0 U 

0.40 


0. 10 
0.14 


i 

ALL CATEGORIES COMBINED 


46.75 


4 . 75 , 


0.10 



FIGURE 2-11 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: . 
STUDENT UNDERPAYMENT 
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Category 

> 


Estimate 
($ millions) 


Standard 
• Error 


Coefficient 
of Variation 


'TYPE OF CONTROL 






< 


Public 

Private 

Proprietary 


82.95 
17.19 
• 25.63 


12.55 
3.15 
6.27 


0.15 
0.18 
0.24 


TYPE OF INSTITUTION 






• 


Four-Year 
Two -Year 
. Other 


60.70 
44.64 
20.43 


9.55 
9.16 
6.18 


0.16 
0.21 
.0.30 


GRANT TYPE 








Independent 
Dependent 


52.07 
. 73.70 


9.09 
8.31 


0.17 
0.11 


* 

^VALIDATION STATUS 








Non-Validated 
Validated 


119.89 
5.88 


13.14 
0.65 


0.11 
0.11 


ALL CATEGORIES COMBINED 


125.77 


'•13.31 


. 0.11 



FIGURE 2-12 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

CASE OVERPAYMENT 
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Category 


Estimate 
( $ millions) 


Standard 
Error 


CQef f icient 
of Variation 


✓ 

TYPE OF CONTROL 






%■ 


ruDl 1C 

Private 
Proprietary * 


108 . 91 
49.95 
11.54, 


28.44 ' 

12.61 

13.33 


0.-26 

0.25' 

1.16 


TYPE OF INSTITUTION 






jr 


Four —Year 

Two-Year 

Other . < 
** 


1 *3A Oft 

32.34 * 
3 • % 9 


25.39 

17.42' 

11.48 


0. 19 
0.54 
2.88 


GRANT f TYPE 


» 




J 

♦ ° * 


Independent 
Dependent 


75.^70 
94.70 


M8.90 
. 19.18 


0.24 
■ 0.20 


VALIDATION STATUS 








Non-Val ldated 
Validated 


_L 0 _L . 0 o 

8.77 ~ 


2.24 


0. 19 ' 
• * °0.26 


ALL CATEGORIES COMBINED 


170.40 


33.00 


0.19 
■ — H 


0 


FIGURE 2-13 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES': . 
NET INSTITUTION ERROR - 



J- i * 



» 

Category 


Estimate 
($ millions) 


Standard 
Error - 


^ " : — I 

\ 

Coefficient 
pf Variation^ 


TYPE OF CONTROL 


/ 

230.96 
61.40 
65.34 


• 


' \ 


Public 

Private 

Proprietary 


31.09 
13.08 
, 14.26 


\.13 
6.21 
0.22 


TYPE OF INSTITUTION 








Four-Year 
Two-Year * 
Other 


199.01 
113.00 
| 45.70 


' 25.33 
21.43 
12.29 


a. 13 

0.19 
0.27 


grant; TYPE 








Independent • 
Dependent 


173. SO 
183.91 


19.10 
18.33 


O.lf 
0.10 


VALIDATION STATUS 
« 






t 


. Non-Validated 
•^Validated /% 


342'. 01 
• , 15.70 


3(3.97 
2.J.4 


0.09 
0.14 


- ALL CA^fiGORIES COMBINED 


. 357.71 


32.38 . 


<f.09 



FIGURE 2-14' 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
, ABSOLUTE INSTITUTION ERROR . , * . 



Category 



Estimate 
($ millions) 



Standard Coefficient 
Error , of Variation 



TYPE^OF CONTROL 

Public 

Private 

Proprietary 



TYPE OF INSTITUTION 

Four-Year 
Two- Year 
Other 



GRANT TYPE 



Independent 
' Dependent 



VALIDATION STATUS 

Non-Validated 
Validated 



L CATEGORIES COMBINED 




131.79 
57.61 
'27.29 



155.32 
44.95 
K.42 



90.77 
125.92 



208. 19 
8150 



216. 69 



14.26 
10.54 
7.50 



16.06- 
7.09 
4.27 



12.18 
11.32 



"16.99 
1.07 



17.09 



0.11 
0.18 
0.27 



0.10 
0.16 
0.26 



0.13 
0.09 



0.08 
0.13 



0.08 



FIGURE 2-15 * 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEG6RIES: 

NET STUDENT ERROR 



2-34 * J 



— — 

Category 


Estimate 
($ millions) 


Standard 
ErVor 


Coefficient 
of Variation 


. TYPE OF CONTROL 






j 


Public 
. Private 
Proprietary 


190.31 
84.46 • 
35.42 


*19.01 
13.32 
9.15 


0.10 

.0.16 
0.26 


TYPE OF INSTITUTION 


"* * 






Four-Year I g 

Two-Year 

Other 


I ' 

224.86/ 
• 62.16N. 
23.17 


420. 76 
»8.74 
. IT 6.31 


0.09 
0.14 
0.27 


GRANT TYPE k 






✓ 


Independent 
Dependent 


109.12 
201.07 


11.84 
15.68 


0.11 
0.08 


VALIDATION STATUS 

' J 

tf6n-Validated 

Validated 


296.07 
14.12 


21. 29 
1.35 


0.07 

. 0.10 . 


ALfc CATEGORIES COMBINED 


*4 310. 19 


21. 73 

f 


0. 07 


' ' ' .-.') 

\ 


FIGURE 2-16 • 
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ESTIMATED SAMPLINGS ERRORS BY SELECTED CATEGORIES: 
ABSOLUTE STUDENT ERROR 



2-35 




• 

• 

Category 


Estimate 
($. millions) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL , 




* 




Public 


240.09 


33. 33 


0.14 


Private 


•106.99 


- 19. 34 


0.18 


Proprietary 


38.88 


" 15. 13 


0. 39 


TYPE OF INSTITUTION 








Four-Year 


289.56 


29. 49 


0.10 


Two-Year 


76. 41 


20. 61 


0. 27 


Other 


19.99 


11. 83 


0.59 


GRANT TYPE 






* 


Independent 


165. 62 


23. 96 


0. 14 


Dependent 


220.33 


22. 24 


0~210 


VALIDATION STATUS 








Non-Val idated 


368.71 


36. 36 


'o.io 


Validated 


17.24 


2. 24 


0.14 


ALL LATfcjbUKlfcjb LUWdHNCjU 


JO J, 7 O 


~\1 50 


0 10 


# 


FIGURE 2-17 







ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

NET CASE ERROR * 



5: 

2-36 

ERIC 



c 



Category 


Estimate 
($ millions) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 




C 




Public 


405.99 


40.89 


0.10 


Private 


14.1.37 


21.98 


0.16 


Proprietary 


90.15 , 


19.34 


0.21 










TYPE OF INSTITUTION 








•Four-Year 


410.96 


34.86 


0.08 


Two- Year 


, 165. 69- 


26. 36 


0.16 


Other 


60.86 


16.09 


0.26 


i 




• 












Independent 


269.77 ' 


23.64 


0.09 


Dependent 


36.fr. .75 

* • 


'25.32 


■ 0.07 


VALIDATION STATUS 
# 








Non-Validated 


608.51 


39.66 


0.07 


Validated 


29.01 


, 2^55 


0.09 


ALL CATEGORIES COMBINED 


637.51 


41.07 


0.06 











./ 

J 9 FIGURE 2-18 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
ABSOLUTE CASE ERROR 



Category 


Estimate 
(Thousands*) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL ' 


c 






Private 
Proprietary 


465. 91 
72.15 
59. 54 


56. 58 
15.32 
13.69 


0.12 
0.21 
0.23 


TYPE OF. INSTITUTION 








r OU L X cdL 

Two-Year 
Other * 


338.31 
216.30 
* 42.99 


36.19 
46.19 
11.56 


0.11 
0. 21 
0.27 


GRANT TYPE 








Independent 
Dependent 


285. 26 
312.34 


36.19 
26.58 


0.13 " 
0.09 


VALIDATION STATUS 

Non-Validated 
Validated 


boy • ui 
28,59 


D Z * 0 / 

3.54 


0 09 
0.12 


ALL CATEGORIES. COMBINED 


597,60 


55,25 


0.09 



FIGURE 2-19 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES Wm INSTITUTION OVERPAYMENT 



J' 



0 



2-38 



4 

Category 


Estimate 
(Thousands) 


Standard 
Error * 


* 

Coefficient 
of Variation 


TYPE OF CONTROL 








Public 

Private 

Proprietary 


299/70 
17.13 
56. 44 


43.28 - 

3.87 
11.42 


0.14 
0.23 
0.20 


TYPE OF INSTITUTION 






« 


Four-Year 

Two-Year 

Other 


132.00 
200. 15 
41.11 


23.75 
38.78 " 
10.22 


£ 

0.18 , 

0.19 

0.25 


GRANT TYPE ; „ 


• ■ 






Independent 
Dependent 


185.99 . . 
187. 28 


• 28.38 
23.55 


. 0.15 
0.13 


VALIDATION STATUS 




♦ < 

• 




Non-Validated 
Validated 


357.28 
15.99 


42.50; " 

1. 69 


"0.12" 
0.11- 


ALL CATEGORIES COMBINED^ 

— 


373.27 


i 

43.36 


0.12 



FIGURE 2-20 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH INSTITUTION UNDERPAYMENT «. ' 



ERIC 
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Category 


Estimate 
(Thousands) 


Standard 
Error 


Coefficient 
6f Variation 


• TYPE" OP CONTROL 






s 


rUDl 1 C 

Private 

Proprietary 

i 


A 9 1 7 1 

182.25 
61.46 


an r r 

4U « OO 

21.7-2 
14,00 


0 10 
0.12 
0.23 

i 

* 


TYPE OF INSTITUTION 






Four-Year 

Two-Year 

Other 


A 7 Q 7 0 

142.27 
43.47 


J 3 • 0 D 

If .56 
10.74 


0 OR 

0.14 
0.25 

* 


GRANT TYPE 






K 

4 


Independent 
Dependent 


158. 53 
506.90 


14.03 
35.87 


0.09 

0.0-7^ 

\ 


VALIDATION' STATUS 






* • ^ ^^^^ 


INUn val 1 vJ a X- cvJ 

Validated 


635 61 
29.82 


40 . 79 
2.54 


0. 06 
0.09 


^ALL ^CATEGORIES COMBINED 


665.43 


41. 50 


0. p6 



FIGURE 2-21 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH STUDENT OVERPAYMENT^ 



0 



Category ' : ;^ 

< 


.Estimate 
(Thousands) 


Standard 
* Error 


Coefficient 
of Variation 


TYPE OF CONTROL 

* * 


• 




- 


Public 

Private 

Proprietary 


122.17 
61. 42 . 
19. 49 


14. 52 
10.80 
8. 50 


0.12 
0.18 
* ^0.44 


TYPE OF INSTITUTION 


- 






Four-Year 
Two-Year 
Other , 


15-0.28 
36. 82 
15.98 


14.51 
7.37 
8.29 


o:io 

0.20 
0.52 


GRANT TX'PE 








Independent 
Dependent 


29. 60 
173. 47. 


4. 95 
16.12 

* 


0. 17 
0. 09 


VALIDATION STATUS 

Non-Validated , 
Validated 


191.14' 
11.94 


16. 81 
1. 57 


0.09 
0..13 


ALL CATEGORIES^ COMBINED 


203.08 


17. 47 


0. 09 



FIGURE 2-22 



ESTIMATED SAMPLING ERRORS" BY SELECTED CATEGORIES: 
CASES WITH STUDENT UNDERPAYMENT 



Category 


Estimate 
(Thousands) 


^Standard 
Error 


Coefficient 
of Variation 


TYPE OF LUNIKUL 








Public 

Private 

Proprietary 


803.14 
239.86 
98.83 


72.79 
28. 59 
21. 1-9 


0.09 
0.12 
0.21 


TYPE OF INSTITUTION 








Four -Year 
• Two -Year . 
Other 

V „ 


75.3.20 
, 320.83 
67.80 


52.76 
52. 33 
17.79 


0.07 
0.16 
0.26 


4 

GRANT TYPE 








Independent 
Dependent 


401.22 • 
■ 740.62 


39. 56 
43.21 


0.10 
0.06 


VALIDATION STATUS 








Non-Validated 
V^l idated 


1,089.30 
• 52.54 


4.32 


UtUO 

0.08 


ALL CATEGORIES COMBINED 


1,141.84 


66.85 


0.06 




FIGURE 2-23 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH CASE OVERPAYMENT 



t 



Z-42 



Pi . 1 " * 



r 



ERIC 



\ 



J 



Category ( 


Estimate 
(Thousands) 


Standard 
* Error 


Coefficient 
of Variation 


TYPR OP fOMTPOT. 




4 

V 




Public 

Privatg. 

Proprietary 
« 


367.26 
71.18 
52.60 


45.39 
11.21 
11. 86 


0.12 
0.16 
0. 23 


XlrEi Uf 1 LNo 1 1 1 U 1 1UIN 








Four-Year 
Two-Year 
• Other 


242.55 
207.10, 
41.38 


27.71 
39. 29 
li.41 


/ 0.11 
/ 0.19 
/ 0.28 


GRANT TYPE 








Independent 
Dependent 


183.66 
307.39 


28.27 
26.98 


0.15 
0.09 


VALIDATION STATUS 








Non-Val ida ted 
Validated 


466. 26 
24.78 


44 . 09 
2. 28 


0. 09 
0.09 


ALL CATEGORIES COMBINED 


491.04 


45.12 


0.09 



FIGURE 2-24 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH CASE UNDERPAYMENT 



2-43 




Estimate Standard Coefficient 
Category Dollars Error of Variation 



TYPE OF CONTROL 

Public 365 36 0.10 

* Private 772 73 0.09 

Proprietary 646 91 '0.14 



TYPE OF INSTITUTION 

Four-Year 492 41 0.08 

Two- Year 336 49 0.15 

Other 578 119 0.21 



GRANT TYPE 



- 4 



independent < 437 40 0.09 

Dependent ■ 446 * 37 0.08 



VALIDATION STATUS 

Non-Validated 443 34 0.08 

Validated 428 \ 39 0.11 



ALL CATEGORIES COMBINED 442 34 0.08 




FIGURE 2-25 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE INSTITUTION OVERPAYMENT. 



2-44 



^aiiegory 


Estimate 
( Dollars ) 


Standard 
Error 


Coefficient 
of Variation 


ixrCi Ur LUNIKUL 




■ 




Public 


382 ■ 


17 


0.d4 


Private 


390 


34 


0.09 


jr it upr i e iiary 


DID 


61 


0 . 12 


Ixrfci Ur INSTITUTION 






• 


Four-Year 


396"' 


20 


0.05 


Two-Year 


376 


29 


0.08 


Uuner 


\ A C C 


C A 

o4 


0 . 12 


GRANT TYPE 








Independent 


' 630 


44 


0. 07 


Dependent 


323 


13 


0 . 04 


VALIDATION STATUS 






-a 


v_/ 41 vai iUu LCU 


397 


17 




Validated 


379 


24 


0.06 


ALL CATEGORIES' COMBINED 


396 


17 


0. 04 



FIGURE 2-26 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE STUDENT OVERPAYMENT 



\ 
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Category 


Estimate 
(Dollars) 


Standard 
Error 


Coefficient 

of Variation 
** 


TYPE OF CONTROL 






* 


PublfC 

Private 

Proprietary 


402 
518 
653 


22 
40 

i 51 


0. 06 
0 . 08 
. 0.08 


TYPE OF INSTITUTION 








Four-Year ^ 

Two- Year 

Other 


465 
377 
596 


24 
35 
52 


. 0.05 
0 . 09 
' 0.09 


GRANT TYPE 




• 




Independent 
Dependent v 

< 


543 
397 

< 


37 


0 . 07 4 
0.05 


VALIDATION STATUS 








won— vai iaa tea 
Validated 


449 

440 , 


21 
29 


0.05 
0.07 


ALL CATEGORIES COMBINED 


448 


20 


, 0.05 




FIGURE 2-27 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE CASE OVERPAYMENT 



: X/ 

Category „ 


Estimate 
(Dollars) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 






• * 


Public 


204 


19 


0. 09 


Pr ivate 


334 


59 


"p. 18 


Proprietary 
• 


- 477 


60 


0.13 


TYPE OF INSTITUTION 

t 




• 




Four-Year 


246 


35 


0.14 


Two-Year 


^ 201 


17 


0:09 


Other 


507 


67 


0.13 


GRANT TYPE 


A 






Independent 


J 264 


24 


0.09 


Dependent 


238 


.24 


0.10 . 


VALIDATION STATUS 








Non-Validated 


252 


20 


0.06 


Validated 


217 


28 


0.13 



ALL CATEGORIES COMBINED 251 19 * 0.08 



j 

FIGURE 2-28 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE INSTITUTION UNDERPAYMENT 



2-47 



ERIC 



* 

Category 


Estimate 
(Dollars ) 


Standard 
Error 


Coefficient 
of Variation 










Public 

Private 

Proprietary 


240 
219 
208 


20 
' 24 
31 


0.08 - 

0.11 

0.15 


rmrDD OP TMC r PT r T 1 TT r T 1 TnM 


-• 






Four-Year 

Two-Year 

Other 


231 
234 
211 


18 

24 
35 


0.08 
0.10 
0.17 


GRANT TYPE 




9 




Independent 
Dependent 


310 
217 


46 
15 


0.15 
0.07 


VALIDATION STATUS , 








Mnn-Val idated 
Validated 


230 

] 236 


14 . 
21 


0.06 
0.09 


# ALL 'CATEGORIES COMBINED 


230 


13 


0.06 



FIGURE 2-29 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

AVERAGE STUDENT "UNDERPAYMENT .*K.-*£ 

. ' . ' •/ • • 
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Category 


Estimate 
(Dollars) 


Standard 
, Error 


Coefficient 
of Variation 


TYPE OF CONTROL 


• 






Public 


226 


16 


0.07 


Private 


242 


23- 


, 0.10 


Proprietary 


487 


59 


- 0.12 


TYPE OF INSTITUTION 




- 




Four-Year 


250 


20 


0.08 


Two -Year 


216 




0. 08 


Other 


494 

*■ 

t 


\ 71 


, 0.14 


GRANT TYPE 


1 

1 

1 


. \ 

i 


> 


Independent 


1 

284 


24 


0. 08 


Dependent 


240 


16 


0.07 


VALIDATION STATUS 








Non-Validated 


257 


15 


0. 06 


Validated 


237 


1 A 

20 


a a a 

0.09 


ALL CATEGORIES COMBINED 


2 56;. 


► 15 


0.06 



FIGURE 2-30 <=^ 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
" AVERAGE CASE UNDERPAYMENT 

9 

/ 



Category 


Estimate 
(Dollars) 


- 

Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








' Public 
Private 
Proprietary 


65 
117 
61 


17 
25 
65 


0. 26 
0.22 
1.07 


TYPE OF INSTITUTION 








Four-Ye ar 
Two -Year 
Other 


92 
46 
30 

r 


18 
24 
83 


0.20 
0. 52 
2.75 


*GRANT TYPE 


\ 






Independent 
-Dependent 


\86 

. > 


21 • 
14 • 


) 0.-25 
' 0.20 


VALIDATION STATUS 








Non— vai laatea 
Validated 


81 


14 
20 


0. 20 
0. 24 


ALL CATEGORIES COMBINED 


74 


14 


0.1J 



I 



FIGURE 2-31 




ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE NET INSTITUTION ERROR 

I 
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Category 



Estimate 
(Dollars) 



Standard 
Error 



Coefficient 
of Variation 



TYPE OF CONTROL 



Public 

Private 

Proprietary 



138 
144 
344 



14 
25 
42 



0.10 
0.18 
0.12 



TYPE OF INSTITUTION, 

Four-Year 136 16 0.12 

Two-Year 160 • 20 0.12 

Other 347 39 0.11 



GRANT ...TYPE 
Independe 

Dependent '- 130 13 0.10 



Independent 197 17 0.08 



VALIDATION STATUS 

Non-Validated ■ 156 13 0.08 

Validated . 145 18 0.12 

ALL CATEGORIES COMBINED 156 13 0.08 



FIGURE 2-32 

ESTIMATED SAMPLING ERRORS BY SELECTEE CATEGORIES: 
AVERAGE ABSOLUTE INSTITUTION ERROR 




2-51 f;* 



« 

> 


• 

Category^ 


Estimate 
(Dollars) 


Standard 
Error 


Coefficient 
of Variation 




— < * — 

^YPE OF CONTROL 






* 


f 

t 


Public 

Private 

Proprietary 


7a 

132" 
139 


7 
16 
33 


0.09 
0.12 
0. 24 . 




TYPE *OF INSTITUTION 










Four-Year 

Two-Year 

Other 


. - 106 
~ 63 
115 


9 ' 
9 
30 


0.09 
0.15 
0.26 




GRANT TYPE 










Independent 
Dependent 


101 
88 


14 
6 


0.14 
0.07 


I 


VALIDATION STAJJJS 










Non-Validated \ 
Validated 


94 
78 


7 
9 


0. 08 
0.11 


ft 


ALL CATEGORIES COMBINED 


93 


•7 


y cos 



FIGURE 2-33^ 

' ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE NET STUDENT ERROR 



< 



U 4 



9 

ERIC 



2-52 



Category 


Estimate 
(Dollars) 


Standard 
Error 


J 

Coefficient 
of Variation 


/TYPE OF CONTROL 






• 


Public 


113 


8 


0.07 


Private 


194 


17 


0.09 


* Proprietary 


180 


28 
* 


0. 16 


TtfPE OF INSTITUTION 




\ 




Fodft/tfear 


153 


10 


0.07 


TwoVrYear 


87 


10 


.0.12 


Other 


162 


21 


0.13 


GRANT TYPE % 








Independent 


122 


14 


* 0.12 . , 


Impendent 

• 


141 


7 


0.05 


VALIDATION STATUS 




9 




~ t 

Non-Validated 


! 

134 


8 


0.06 -' 


Validated 


129 


10 


0.08 


■v. 

ALL CATEGORIES COMBINED 

0 


133 


8 


0.06^ 



FIGURE 2-34 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE ABSOLUTE STUDENT ERROR 



Category 


EJstimate 
(Dollars) 


Standard 
Error 


Coefficient 
of Variation 


TxJrfci Ur LUIN 1 i\UJj 








Public 

Private 

Proprietary 


142 
250 
204 


18 
3Q 
57 


0.12 
0.28 


1 xrlj Uf iiiOiiiuiivii 








Four-Year 
Two-Year 
? Other 


198 
107 
151- 


20 
27 
68 


0.10 
0.25 
0.45 


GRANT TYPE 








Independent 
Dependent 


186* 
155 


27 
15 


0.15 
0'.09 


VALIDATION STATUS 








Non-Validated 
Validated 


lOO 

157 


16 


0. 10 
0.12 


ALL CATEGORIES COMBINED 


167 


16 


0.10 




FIGURE' 2-35 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: ^ 
AVERAGE NET CASE ERROR 



Category 


Estimate 
(Dollars) 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL / 








Public 


240 


15 


0. 06 


Pr ivate 


330 


28 


0.09 


Proprietary 


473 


39 


0. 08 


TYPE OF INSTITUTION 




• 


• 


Four-Year 


281 


' 18 


0.07 


Two- Year 


23'3 


22 


0.09 


Other t 


459 


31 


0.07 


GRANT TYPE 


• 


• 




r 

Independent 


• 303 


2.2 - 


0.07 


Dependent 
< 


^ 259 


13 


0.05 


VALIDATION STATUS 






• 


Non-Validated 


277 


15 


0.05 


Validated 


265 


17 


0.06 


ALL CATEGORIES COMBINED 


276 


14 


0.05 



FIGURE 2-36 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
AVERAGE ABSOLUTE CASE ERROR 



Category 



Estimate 



Standard 
Error 



Coefficient 
of Variation 



TYPE OF CONTROL 

Public- 22,797 8,543 0.37 

2,265 1,189 . 0.52 



Private - • : - _r A i 

Proprietary 3,513 . 1,803 0.51 V 



TYPE OF INSTITUTION 

Four-Year 

Two-Year 

Othtr 



GRANT TYPE 



Independent 
Dependent 



VALIDATION STATUS 

Non-Validated 
Val idated 



ALL CATEGORIES COMBINED 



21,302 
5,102 
2,153 



12,714 
15,862 



27,203 
1,37.2 



28,575 



8,280 
2,415 
1,525 



5,333 
4,058 



8,357 
462 



8,562 



0.39 
0. 48 
0.71 



0. 42 
0.26 



0.31 
0.34 



0. 30 



FIGURE 2-37 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH ACADEMIC PROGRESS ELIGIBILITY ERRORS 



*7 - 



0 

ERJC 



2-56 







Standard 


Coefficient 


Category 


Estimate 


Error 


of Variation 


TYPE OF CONTROL 








FUD1 1C 


O ICO 

2 , 1 b J 


1 TOO 

1 f 183 


0 . 55 


Dv* "J »»2 +- o 


u 


u 




Proprietary 


' 112 


112 


1.00 


TYPE OF INSTITUTION 


- 




\ 


r our XcdL 


i *3 £ n 


y o z 


U . / 1 


Two-Year 


793 


689 


0.87 


utiner 


11 I 


1 1 o 

112 


1 . U U 


GRANT TYPE 


• 






Independent 


1,473 


968 


0.66 


Dependent 


q n o 

793 


689 


0.87 


VAblDAllON STATUS 








Non-Validated 


2,040 


1,178 


0.58 . 


Validated 


225 


159 


0.71 


ALL CATEGORIES COMBINED 


2,265 


l',189 


0.52 



FIGURE 2-38 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH ENROLLMENT STATUS ELIGIBILITY ERRORS 



7"> 



5 



Category 


Estimate 


Standard 
Error 


Coef f icie'nt 
of Variation 


TYPE OF CONTROL 








Public 

Private 

Proprietary 


680 

U W \J 

0 
0 


680 
0 
0 


1. 00 

• 
• 


TYPE OF INSTITUTION 








r OU L X ca L 

Two-Year 
Other 


680 
' 0 
0 


680 
0 
0 


1.00 , 

• 
• 


GRANT TYPE 








Independent 
Dependent 


0 

680 


0 

680 


1.00 


VALIDATION STATUS 








INOn Vol lUd CcU 

Validated 


680 
0 


680 

o- 


1.00 

• 


ALL CATEGORIES COMBINED 


680 


680 


1.00 



FIGURE. 2-39 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES 
CASES WITH COURSE LENGTH ELIGIBILITY ERRORS 



9 

ERIC 



2-58 



0 



Category 


Estimate 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








Public 


1,360 


962 


0, 71 


Pr ivate 


0 




• 


Proprietary 




0 


• 


TYPE OF INSTITUTION ^ 








Four— Year 


C Q H 
DOU 


DO U 


1 . U U 


Two-Year 


680 


680 


1.00 


Other 


U 


A 

0 V 


• 


GRANT TYPE 








i nuepe nae 


u 


u 




Dependent 


1,360 


962 


0.71 


VALIDATION STATUS 








Non-Validated 


1,360 


962 


0.71 


Validated 


0 


0 


• 


s 




> 




ALL CATEGORIES COMBINED 


1,360 


962 


0.71 



FIGURE 2-40 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH DEGREE STUDENT ELIGIBILITY 'ERRORS 



4 

( 



"' "ft 

ERJ.C 

a 



Category 


Estimate 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 


- / ' 






P.ublic 

Private" 

Proprietary 


2,720 
1,360 
0 


1, 388- 
962 
0 


0.51- 
0.71 

• 


TYPE OF INSTITUTION 








Four -Year 
Two -Year 

Other 1 


' 2,720 
1,360 

0 


1,360 
1,*001 
0 


0.50 
0.74 

• 


GRANT TYPE 








Independent 
Dependent 


2,040 
2,040 


1,210 
1,178 


0. 59 
0.58 


VALIDATION STATUS 
Validated 


4,081 
0 


1,689 

0 . 


i 

0.41 

• 


ALL CATEGORIES COMBINED 


4,081 


* 1,689 


0.41 



FIGURE 2-41 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH B.A. ELIGIBILITY ERRORS 
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Category 


Estimate 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








Public 


51,453 


21,947 


0.43 


Private • 


22,911 


7,736 


0.34 


Proprietary 


18,535 


8,017 


0.43 


* 

TYPE OF INSTITUTION 








Four-Year 


61,903 


23,231 


0.38 


Two -Year 


16,542 


6,177 


0.37 ' 


Other 


14,454 


6,455 


0.45 


GRANT TYPE 




• 




Independent 


35,691 


12,933 


0.36 


Dependent 


57,207 


15,008 


0.26 


VALIDATION STATUS 






• 


Non-Validated 


87,951 


23,697 


0. 27 


Validated 


4,948 


1,783 


0. 36 


ALL CATEGORIES COMBINED 


92,899 


25,114 


0.27 



FIGURE 2-42 ^ 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH A.E.P. MISSING 



* 



Category 


Est imate 


Standard 
Error 


Coefficient 
of Variation 


TYPE OF CONTROL 








r UD1 1 C 

Private 
Proprietary 


66 , 449 
24,951 
10,876 


15,260 
6,552 
4,380 


0.23 
0.26 
0.40 


TYPE OF INSTITUTION 








Two-Year 
Other 


63 , 386 
29,374 
9,516 


12,453 
7,628 
4,209 


0. 20 
0.26 
0.44 


GRANT TYPE 








Independent 
Dependent 


57,637 
44,639 


10,204 
7,690 


0.18 
0.17 


VALIDATION STATUS 








Non-Validated 
Validated 

J 


98,340 
3,936 


14, 653- 
854 


0.15 
0.22 


ALL CATEGORIES COMBINED 


102,276 


15,042 


0.15 


v 

* 


FIGURE 2-43 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES MISSING F.A.T. 



v-atiegory . 


/ 

DSTiiiuaTie 


otanaar a 
Error 


uoet ticient 
or variation 


.TYPE OF CONTROL 








Public 


905 


698 


0.77 


Pr i vate 


U 


u 


• 


Proprietary 


A 

Q 


U 


• 


TYPE OF INSTITUTION 


«* ■ 


- 


- 


Four-Year 


793 


689 


0.87 " 




11Z 


i i o 


* 1 • U U 


Other 


0 


0 


• 


GRANT TYPE 








inaepe naen tl 


0 o u 


0 o u 


i n n 
1 . uu 


Dependent 


225 


159 


0.71 


VALIDATION. STATUS 








Non-Validated 


680 


680 


1. 00 • 


Validated 


225 


159 

* • 


0.71 


ALL CATEGORIES COMBINED 


905 


698 


0 . 77 



FIGURE 2-44 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH- LOAN DEFAULT 



/ 



Category 


• 

Estimate 


Standard 
Error - 


Coefficient 
of Variation 


TYPE OF CONTROL 








Public 

Private 

Proprietary 


1 / JOU 

0 
0 


962 

ZJ \J 

0 

0 . 


0.71 

• 
• 


TYPE OF INSTITUTION 








Four-Year 

T^wo-Year 

Other 


i "3 <c n 

/. 


962 
0 
0 


0.71 


GRANT TYPE & 








Independent 
Dependent 


680 
680 


630 
680 


1.00 
1.00 


VALIDATION STATUS 


• 






Non-Validated 
Validated 


1,360 
0 


96g 
0 


U . / 1 
• 


♦ 

ALL CATEGORIES COMBINED 


1,360 


962 


0.71 



FIGURE 2-45 
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ESTIMATED* SAMPLING ERRORS BY SELECTED CATEGORIES: 
CASES WITH CITIZENSHIP ELIGIBILITY ERRORS 



9 

ERIC 



2-64 



0 



SURVEY RESPONSE RATES AND POTENTIAL NONRESPONSE BIAS 

In the following section we discuss response rates for the 
survey of parents and students and review £he existing data for 
potential nonresponse bias. As shown in~ the ,fol lowing section, 
the response rate was high and any problen with nonresponse , bias 
is likely to be of minor significance. 
Response Rates for the Student/Parent BEOG Survey 

The fieldwork carried out by Westat resulted in completed 
questionnaires sufficient to verify all application forgi entries 
for 90 percent of the sampled grants. Details concerning this _ 
respot^e rate, as well as an alternative calculation, are 
included below. 

Defining response rates for the BEOGs Quality Control study 
presents a unique problem because a case or grant could involve 
two separate questionnaires. For a dependent student, it is 
necessary to have a completed questionnaire from both the student 
and parent in order to verify all the application entries. Fur- 
thermore, since both signed the application, their 'cooperation is 
mandatory . 

For independent student grants, it is only necessary to have 
a completed student questionnaire to completely verify the appli- 
cation entries. Those entries concerning parental support which 
are used to establish independent status are verifiable using the 
student questionnaire responses. 



2-65 °' J 
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The response rates shown in Figure 2-46 are based on three 
general disposition categories — completed; unavailable for inter- 
view; and refusers, avoiders, breakoffs, etc. Figure 2-47 lists 
w^Lch detailed disposition codes have been combined in forming 
t'hese three general categories. 

The rates are all based on the number of original sampling 
points and questionnaire dispositions presented in Figure 2-47. 
The 3 response rates for dependent student grants are all above 
90 percent; nearly 95 percent for student questionnaires; and 
almost 94 percent for parents. . For 91 percent of the dependent 
student grants we were able to get completed interviews .with both 
the parent and student. 

Response rates for ^independent student grants are not as 
high, but for students we were able to complete over 87 percent 
of the interviews. Since the .parent interview was voluntary for 
"independent student grants, the degree of cooperation was 
expected to be low. These expectations were borne out as indi- 
cated by the 61 percent response rate for independent parent _ 
questionnaires. Of course, the percent of cases for which we 
completed a student and parent questionnaire also reflects this 
low level of cooperation, as indicated by the 58 percent paired 
completion rate. As indicated by the footnotes in Figure 2-46,.. 

many parents were deceased, out of the- country, or could not be 

. * - 

located. 

The final two response rates are based on two methods of ^ 
combining dependent and independent grants. The first entry is 
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. Nonrespondents 


Grant type and form of 
questionnaires 


Completed 


Unavailable 
for 
Interview 


A Refusers, 
• fw Avoiders, 
Breakoffs, etc. 


Dependent Student Grants: 
Student Questionnaire 
Parent Questionnaire 
Student and Parent 


94.6 
93,6 
91.4 


1-.0 
1.9 
2.2 


4.4 
4.4 
6.4* 


Independent Student Grants: 
Student Questionnaire 
Parent Questionnaire 
Student and Parent \ 


87.4 
61.0 
58.4 


1.0 
15.6** 
15.3* 


11.6 

23.4*** * 
26.3 


Dependent and Independent Grants: 
Student and Parent 
Student and Parent for 

Dependent Student Grants 
* and Student Questionnaire 
for Independent Student 
Grants . * 


79.5 
90.0 


6.9 
1.8 


13.6* 
■ 8.3* 



FIGURE 2-46 



RESPONSE "RA^&S 




w grant is included in this category if parent, student, or both were 
/refusers, avoiders, breakoffs, etc. 

**mzur±y 52 percent (137 ca^es) of this 15.6 percent consist of deceased 
♦ parents; 36 percent (96 cJ^es) consist of parents who are out of the 
country. 

***0ve* 41 percent (162 cases) of this 23.4 percent consist of parents who 
could not be located. , ~ 
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A. Dependent Student Grants 





Student Questionnaire Disposition 






Nonrespondents 

- • > 




. Parent Questionnaire 
* Disposition 


Completes 


Unavailable for 
interview 


Refusers, 
Avoidets, Break- 
offs, etc. 


Totals 


Completes 


2,732 


15 


% 53 


2,800 


Nonrespondents : Unavailable 
Nonrespondents : Refusers , 
Avoiders, Breakoffs, etc. 


57 


12 
2 


5 
74 


57 
133 


Totals 


2,829 


29 


132 


2,990 



7 



B. Independent Student Grants* 



Parejit ^Questionnaire 
' Disposition 



Student Questionnaire Dispositipn 



Completes 



Nonrespondents 


Unavailable for 


Refusers, 


Interview 


Avoiders, Break- 




loffs, etc. , 


4 


40 


i 

# 9 


10 


* 4 


145 


17 


195 







Completes 

Nonrespondents : Unavailable 
Nonrespondents : Refusers, 
Avoiders, Breakoffs, etc. 



985 

245 
245 



Totals 



1,475 
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FIGURE 2-47 
PARENT QUESTIONNAIRE DISPOSITIONS, BY 



\ 



J 



General 
t Category 


'Detailed Disposition 
Categories 


completes 

- 


11 


— 


Interview Completed 




Unavailable for 


14 


— 


Extended illness 




Interview 


20 




Sampling error 






21 




Out of country 






22 


- 


Deceased 






35 




Away for field period 




Refusers, Avoiders, 


12 




Maximum Calls 




Breakoffs, etc. 


13 




Cannot Locate: Address from 


School 




15 




Ref usal/Br^kof fi 






16 




Avoider 






17 




Language Problem 






18 




Other 






31 




-Cannot Locate: No address from School 




32 




Other member refused 






33 




Refused because student quit 






34 




No involvement with student, 


indepen- 








dent parents only. 





FIGURE 2-48 / 
DISPOSITION CATEGORIES 
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based oh the demanding paired completion for both grant types. 
Its value of about 80 percent reflects the problem of getting 
independent parents to cooperate . 

The last entry of 90 percent is most reflective of the study 

«■ 

requirements . Here we used the paired completes for dependent 
student grants and student completes for independent student 
grants. Thus, for 90 percent of . the^ grant s we have questionnaire 
response .sufficient to verify all application form entries. 
Analysis of Nonrespcmse Bias 

The 'purpose of this analysis of nonresponse in the survey of 
BEOG grant recipients is to determine the possible nonresponse 
bias in estimating program-wide grant error amounts. Even though 
this survey, as noted in the previous section, achieved very high 
parent and student response rates (90 percent overall)/^ careful 
analysis of the possible impact of the missing nonrespondents is 
important to the credibility of the study's eventual findings and 
conclusions . - 

The fieldwork carried out by Westat resulted in completed 
questionnaires sufficient to verify all application form entries 
for 90 percent of the sampled^grants . This composite rate, can be 
broken down into separate rates for independent and dependent 
student grants as follows; Questionnaires were completed for 
both students and parents for ^1.4 percent of the dependent stu- 
dent qrants. At least a student questionnaire was completed for 
87; 4 percent of the independent' student grants. ' \ 



2-70 



* Defining response rates for the BEOG Quality Control study, 
presents a unique problem because a case or qrant could, involve 
two separate questionnaires.! For a dependent student, it is 
necessary to have a completed questionnaire from both the student 
and parent in order to verify all the application entries. Fur- 
thermore, since both signed the application, their cooperation is 
mandatory. For independent student grants, it is only necessary 
to have a completed student questionnaire to completely verify 
the application entries. Those entries concerning parental 
support which are used to establish independent status are veri- 
fiable using Ae student questionnaire ^responses. 

Even with response rates this high, there is the possibility 
that tt\e_ respondents may be quite dif ferent ^fron nonrespondents 
in terns of disbursement error. While error in disbursements .is 
the focus of this Quality Coptrol study, for nonrespondents it is 
not possible to calculate grant error amount because we lack the 
necessary verification data, which is normally obtained from 
.interviewing respondents. However,- we can* test the hypothesis 
that "oonrespondents are students who misrepresented their finan- 
cjat circumstances in order to obtain a substantially larger 
grant disbursement; '.amount . " - ft this hypottffesis were true, we 
would expect nonresponderrts to have received higher disbursements 



A In addition, «_^ta collected through th^-S^udent Record 
Abstracts [SRA} could provide data sufficient: to calculate a 
jjrant error- 
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than respondents. This could be tested by comparing the average 
disbursenent amount for respondents versus nonrespondents. Hqw- 
ever, data on actual disbursements were not available when this 
analysis was performed. Available data on fall 1980 values of 
the student eligibility index (SEI), scheduled Basic Grant, and 
expected disbursements are expected to be highly corrrelat^ with 
actual disbursement and will be used in this analysis as proxies 
for -actual disbursements. 

In later sections we compare the average values for these 
three proxy variables for respondents and nonrespondents to 
ascertain the extent to which the differences would be symptom- 
atic of misrepresentation of financial circumstances for the 
purpose of obtaining a substantially higher disbursement. While 

ninor\iif ferences do exist, their levels are not sufficiently 

Q 

high to seriously undermine subsequent analysis. Furthermore, 
while nonrespondent .dependent student cases have higher expected 
disbursements than respondents, the opposite is true for indepen- 
dent student cases, e.g., the potential biases are in the oppo- 
site directions. Finally, the average differences are often 
within the tolerances established for the student validation 
procedures.- Thus, the evidence currently available does not 

1 The effective tolerance is about S50 in terms of expected 
disbursement. However, this tolerance is limited to only a 
subset of the application items which affect the grant level. 



support the hypotheses that nonrespondents are receiving sub- 

c 

stantially larger grants than respondents . 1 

In this report we utilize data collected in the fall of 
1980. -Spring 1981 data on enrollment status, actual and planned 
disbursements,, and more recently calculated scheduled entitle- 
ments had already been collected; however, this data set was not 

available for analysis. It is expected that an analysis using 

i 

the more recent data would yield identical conclusions with res- 
pect to nonresponse bias. 
Data Base Description 

The data base for this analysis consists of elements from 
four basic data sources as shown below: 



BASIC DATA SOURCE 

Fall Student Eligibility 
Report [SER] 



Student Questionnaire 
Parent Questionnaire 

t 

Master Sampling File 



ELEMENTS 

Student Eligibility Index [SEI] 
Scheduled Basic Grant 
Expected Disbursement 
Grant Status (Independent Or 
Dependent ) 

Student Questionnaire 
Disposition 

Parent Questionnaire 
Disposition 

Sampling Weight 



Records from these four files were merged according to case' 
identification numbers. The final merged file contains about 
4,700 records, 1 for each originally sampled grant. 



Note that it is still possible that nonrespondents might have 
larger 'g^a^t errors ' than respondents; but" drfta are not available 
to make such f a test. * 



/ 



ERIC 



2-73 



The SEI, scheduled grant, and expected disbursements are" 
taken from the SER submitted by schools last fall. In other 
parts of the main analysis report, the SEI taken from school 
files in the spring is analyzed. As a consequence, average 
scheduled grants and expected disbursements used in this analysis 
will be somewhat higher than their spring counterparts because 
students tend to reduce credit hours or drop out as the academic 
year proceeds. 

The purpose of this analysis is to assess differences in 
SEI, grants, and expected disbursements between respondents and 
nonrespondents? however, these values are oftentimes missing for 
respondents and nonrespondents as indicated in Figure 2-49. 

Figure 2-49 indicates that the chance of missing an SEI, 
grant, or expected disbursement is higher for nonrespondents than 
for respondents. Less than 1 percent of the respondent cases had 
missing SEI: nonrespondents had missing SEI in 1.8 percent of 
the cases. Missing expected disbursements occurred in 25 percent 
of the respondent cases and 40 percent of the noni/espondent 
-cases. These differential missing data rates between respondents 
and nonrespondents somewhat limit the strength, of any conclusions 
which can be drawn from the following analysis. 
Discussion of Results 

In this section the weighted average values for SEI, v sched- 
uled grant, and expected disbursement are presented for respon- 
dents and nonrespondents. 

r \ 
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Data Element 


Presence 


4 












of 


Student Eligibility Index 


Scheduled 


Expected 


Data Element 






Grant 


Disbursement 




Respondents 


Nonrespondents 


Respondents 


Nonrespondents 


Respondents 


Nonrespondents 


Element Present 


4166 


447 


3193 


288 


3144 


275 


Element Missina 


37 


8 


1012 


167 / 


1059 


180 


Total Cases 


4203 


455 


4205 


455 


4203 


455 


Cases with 














% Missing 


0.9 


1.8 


24.1 


36.7 * 


25.2 


39.6 


Data Element 















FIGURE 2-49 
MISSING DATA ELEMENT VALUES 

\ . • 



91 



INCIDENCE OF 



90 



These measures do not represent the ultimate focu^pf this 
Quality Control study: measuring the levels of error in actual 
disbursements is its major purpose . We will not be able to 
measure errors in the above three magnitudes for nonresponSents ; 
thus, we are left with assessing differences in their levels as a 
measure of potential • nonresponse bias. ' 

Figure 2-50 presents information concerning expected dis- 

i 

bursements. The mean expected disbursement is $985.2 as indi- 
cated in the left-most box. • .The next two boxes present the aver- 
age values for respondents and nonrespondents with the circle 
between the*two boxes containing the difference between the 
respective means. Nonrespondents had expected disbursements 
about $28 higher than respondents. Independent student nonres- 
pondents had expected disbursements lower by $32, while dependent 
student nonrespondents had expected disbursements higher by $67. 
None of these three differences is statistically different from 
zero. Furthermore , their levels are not large in a substantive 
sense. The $28 difference is safely within the tolerances used 
in the BEOG validation procedures. 

Average scheduled grants are presented in Figure 2-51. The 
overall difference of $53 and the difference, $89, for dependent 
students are statistically different from zero; however, their 
levels are not much above the $50 tolerance used in the BEOG's 
validation procedures. The difference of $10 for independent 
students is of little consequence. , 
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Respondents 



Mean = 983.0 
N ■ 3J44 

Missing = 1 ,059 






Independent Student 
Respondents 


Mean = 1 ,029.8 
N = 1,096 
Missing = 397 


t 


V 















Independent Student 




Nonrespondents 




Mean 


= 997.9^ 




N 


= 121 




Missing : 


= 88 



Dependent Student 
Respondents 



Mean 954.5 
N = 2,048 

Missing = 680 




67.2 



Dependent Student 
Nonrespondents 



Mean 
N 

Missing 



1 ,021.7 
154 ' 
92 



93 
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FIGURE 2-50 

AVERAGE EXPECTED DISBURSEMENTS BY CASE 
DISPOSITION AND GRANT STATUS 



O < 



l 

co i 



All 



Mean = 1,02a. 9 
N 3,481 
Missing = 1 ,1 79 



0 

ERLC 



Respondents 



Mean = 1,019.5 
N = 3,193 

Missing = 1 ,012 



^53~4*^ 



Nonrespondents 



Mean = 1,072.9 
N = 288 

Missing = 167 





Independent Student 




Respo ndents 


f 


Mean = 1 ,091 .8 




N = 1 ,105 




Missing = 370 














Independent Student 
Nonrespondents 


Mean = 1,082.2 
N .= 129 
Missing = 80 


i 



FIGURE 2-51 

AVERAGE SCHEDULED GRANT BY CASE 
DISPOSITION AND GRANT STATUS 



Dependent Student 
Respondents 



Mean = 975.8 
N = 2,086 

Missing = 642 




Dependent Student 
Nonresponden.ts 



Mean •■ 1 ,064.4 
N 159 
Missing = 87 



( 



The greatest disparities or differences are for the average 
student eligibility index as presented in Figure 2-52. .Here 'the 
'difference for all grant types is 120 points, and for dependent 
students it is equally high. It is interesting that the effect 
pf large differences in SEI is rather strongly mvtigated by the 
grant determination procedures. Many grants and expected dis- 
bursements are not affected by changes in the SEI because of the 
three-part grant determination procedure. 

Since the ultimate objective of the*BEOG Quality Control 
study shoi^ld be disbursement errors, it is reassuring that dif- 
ferences in average expected disbursements are so low. It is our 
opinion that the delta base, is not affected seriously enough by 
potential nonresponse bias to have any substantive impact on pol- 
icy, conclusions. 

^Furfoier Analysis of Potential -Nonre^pofufe . Bias N " 9 

As «we have already stated, we do not feel that the data base 
is seriously affected by nonresponse. To further strengthen this 
conclusion we h^ve performed sensitivity analys.es hs presented in 
the following. If there is no nonresponse bias, as we contend, \ 
inferences drawn from, respondents 1 da'ta*would also be expected to 
hold for nonrespondents. As a result, estimation of totals wilJL 
be based on a straightforward adjustment using this assumption. 

However, there are ' alternative assumptions which can be' made with 

♦ \ 
regard to nonrespondents, as depicted^ in Figure 2-53. 
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GO 



Respondents 



Mean' = . 443.6 
N * * = 4,1 66 
Missing = \ 37 





Independent Student 
Respondents 



Mean 
N 

Missing 



211.6 
1,461 
14 




to 

\ 


• 


1 • , 


* 

• A* 

*• * 






Independent Studfent 
Nonrespondents 


Nonrespondents 




,Mean = 174.1 
N = 207 
Missing = 2 


Mean = 322.6 
N = 447 
Missing = 8 
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FIGURE 2-52 

AVERAGE STUDENT ELIGIBILITY INDEX BY 
CASE DISPOSITION AND GRANT STATUS 



Dependent Student 
Respondents 



Mean = 585.3 
N * = 2,705 
Missing = 23 




Dependent Student 
Nonrespondents 



flean 
N 

Missing 



464.0 
240 
6 



n 1 



SCENARIO' m • 


ESTIMATE OF 
TOTAL NET FRROR {\ Mil 1 Tf)N^ 


AVERAGE ERROR 

AQCMMCn CHD MHKID c cnn imr iit r 


Respondents and nonnsspondents 
have equal net error 




\ 

$192 


Entire value of expected 
.disbursement is in error 


.563 


« 1,011 


Nonrespondents had errors 
equal /to: 


V m 


■> 


95th percentile 

90t+i percentile 
. 75th percentile - V . 
• 25th frer^ntile' 

lQ,th .percentile 
5th percentile 


540 

* , *' 471 
; 1 * n '427 
. * " 402 

- 380 . 


1,126 

838 
326 

• - 0. • ' 
-186 ' - 
-350 
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FIGURE 2-53 

sensitivity of total net error estimate using * 
alternative' assumptions concerning 



^ - nonrespondents 
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\The first entr^ of $4^53 million is based on the assumption 

y 

that average net error is the same for resporrdents and nonr'espon- 
i * 

dents. The second' largest error estimate of $563 million is 
based oh "the assumption that the entire expected • disbursement for 
nonrespondents *is in' error. , * 

a * The largest estimate based on the 96th percentile, $578 mil- 
lion, assumes that the average error for nonrespondents, $1,126, 
exceeds their average expected disbursement—a somewhat unrealis- 
tic assumption. , 

r Overall, the estimate used in t"he main report, $45 3 mil- 
lion, could be underestimated by $110 million only if the average 
error for nonrespondents equaled their entire expected disburse- 
ment. . 

Th4 problem to be avoided is concluding that there is no 

v. 

error when ir^deed there is error, e.g., downward bias. In our 
cape, ythe baseline estimate is high enough that even if it is 
biased downward by .$100 million, the magnitude of the estimated 
error justified thei conclusion that there is significant error in 
the Basic Grant program. \ 

V 
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CHAPTER 3 



DATA COLLECTION AND QUALITY CONTROL 



Data were collected during the spring of 1981 ' f 50m a - 
nationally representative sample of 305 institutions and fr* 
approximately 4,500' BEOG recipients and thej^^e^err€s\ In all, 
eight data sets, displayed in .Figure 3-1 below, were collected. 
This chapter is 1 organized into .three sections. First, the stu- 
dent and p^ent survey, conducted by Westat's nationwide' network 
of interviewers, is "described. Second, the collection of second- 
ary data used to verify student application data is dis'cuss'ed. 
In the final section, the institutional survey conducted by \ 
Advanced Technology is described. ' \ 





Date Se/1- 


Data Source 




Student File 


Student Questionnaire: obtained from In-field interviews 






with sampled students 


2. 


Parent File 


Parent Questionnaire; obtained from in-fleld Interviews 






with parents of sampled students 


3. 


SER File 


Student Eligibility Reports obtained from sampled * * 






institutions * 


4. 


IRS File 


Certified " 1979 IRS 1040 and 1040A forms, obtained through 






releases from sampled students and parents 


5. 


Tax Assessor 


Obtained through releases from sampled students and 




Records 


parents 








6. 


Fl nancl al 


Obtained through releases from sampled students and 




Institution 


' parents 




Records 




7 - 


Institution 


Institutional Questionnaire:. obtained from interviews with 


1 , 
1 


File 


financial aid administrators a.t sampled Institutions 


1 

8. 


Student Record 


Student. Record Abstract: obtained from student file reviews 


* 


Abstract f\\e 


at sampled Insltitutlpns 



FIGURE 3-1 

3E0G QUALITY CONTROL DATA SETS 
AND SOURCES 



STUDENT/PARENT INTERVIEWS 

The following aspects of the field data collection which 

< 

will be discussed below include : 



Questionnaire development and pretest 
' Field management structure 
Interviewer recruitment 
Training 

Field operations issues 

Reporting and the Automated Survey Control System 
Field problems 



Questionnaire Development and Pretest * , 

The basis for the first draft of .the student/parent data 
collection instrument was the 1978-79 questionnaire .revised .by 
recommendations from Department .of Education ^ED' , - Advanced 
•technology, and Westat project staff- Eight drafts of the ques- 
tionnaire wer§ subsequently produced before the final question- 
naires were printed. Pretests of draft II and draft IV provided 
useful information on the effectiveness of the questionnaires. 

The pretest of draft II took place in the week of October 
-22-24. A brief training session was held which included the 
following topics: - - \ 
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Westat questionnaire* format* conventions and general 
interviewing techniques 

Question-by-question specifications fkr the newly 
revised questionnaire K + 



Documentation requirements 
? 

3-2 • 
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Most of the Advanced Technology and Westat BEOG project staff 

interviewed in the pretest ^along with a professional pretest 

interviewer from Westat. Pretest respondents were selected from' 

Montgomery County College, and Howard University. They "and their 

«■ 

parents were contacted b'y telephone by a Westat interviewer who 

i 

explained the 'study, screened the respondent for dependency 
status, and scheduled an appointment. >Nine independent students, 
nine dependent students, nine dependent parents, and one inde- - 
pendent parent was interviewed. 

A debriefing session was held at the end of the week for 
pretest participant^. The following topics were covered: 



• 



Questionnaire administration time 
Flow of interview 

Ease of ""questionnaire use including <inst3?€<;tions 
and format 

Clarity "of questions/ clarity df responses, and 
coding, question by question 

Other issues which participants wished to discuss 



It was very clear from the pretest that: (1) the instrument 
needed to be divided into parent and student questionnaires; (2) 
the format for Section B (verification of application form items) 
needed to be simplified; and (3) the instructions and phrasing of 

4 

questionnaires 'needed to be simplified. 

Draft IV was produced from the results of the pretesting of 
Draft II and from comments by ED, Advanced Technology, and. Westat 
project staff. Important revisions first evident in this draft 
were: (1) the emphasis of the time reference in each question and 
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(2) the creation, of a "problem question" as the first question in 
each instrument/ Draft IV was pretested by the pretest inter- 
viewer (4 interviews) . The results indicated that the instrument 
flowed very well. The revisions which were made, 'to subsequent, 
drafts were 'relatively minor. (If more detail 'about the process 
is desired, see Westat Methodological- Report . ) 

Suggestions for future*BEOG Quality Control .questionnaires 
based on our field experience include: 



• Obtain more information about the role of the financial 
aid officer in c6mpleting application forms. Apparently 
there .&re instances .in which he or she filled-out the 
application for a student. 

# * ' pimit the questions asked of the "parents of independent 
students, Tijose who were not involved in filling out . v 

•the form became annoyed, or thought it was a waste of 
money to conduqt an'in-person interview for so few 
"•questions. 



c 



Field Management Structure 

r 

To efficiently manage the BEOG study interviewing staff and 

coordinate field operations, Ihe continental United States was 

divided into £even supervisory regions. See Figure 3-2 which 

sh6ws ther supervisory region boundaries as well as the number of 

» 

sampled institutions in each state. Supervisors manaqecl the 
interviewers residing in their . region and Reported to the Field 
Director as shown in the diagram^ on the next page. 
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FIGURE 3-2 



SUPERVISORY' REGIONS FOR STUDENT/PARENT INTERVIEWERS 
BY NUMBER OF- INSTITUTIONS IN EACH SUB REGION 



Project director 



Field Director 



#3 Regional Sup. 




#1 Regional Sup. 



#2 Regional Sup^ T~ 

lints. 



^nts'. 



Ints. 



#4 .Regional Sup^ s^* 

I Ints . 



#5* Regional Sup, 



#6 Regional Sup. 



•,#7 Regional Sup. 



^Irits . 



Interviewer Recruitment 

— — / 

Regional Supervisors and the^r assistants began inter- 
viewer recruitjnent *bn. December 3, 1980. Primary foarces for 
recruitment^ in^^uded Westat'.s computerized interviewer file, 
•supervisors \ local contacts, local employment, agencies , and 
when necessarV, newspaper ads. When interviewer 'training began 



on< January 26,* 19^1, 201 interviewees, had been recruited and 
invited to training. An additional 12 1 interviewers were recruited 
and trained in February 1981. The major recruitment effort' was 
.conducted with minimal information about the ^ location of ^sample * 
respondents. During December and January 'the Regional Super- 
visors knew the location of s^amp^ institutions but not the* v 
number or location of individual respondents . from each institu- 
tion. - ' , * 



Of the 213 interviewers recruited, 203 attended training and 
201 successfully completed training. During^training , the inter- 1 
viewers were- invited- to complete a background information form. 
A review of these forms shows that 59 percent of «the interviewers 
had worked on at least 1 prior Westat study and 99 percent had 
previous experience as survey research interviewers. 
Training 

Training for the seven supervisors and six extra trainers 
(community leaders) took place the week of January 12, 1981. The ' 
first three days were spent acquainting the supervisors and other 
trainers with BEOG study materials . (the questionnaires field 
procedures^ and interviewer training materials). The last two 
days were spent training supervisors on their duties (i.e., use 
of the automated reporting system) . Supervisor training was 
conducted in the Westat of f ices in Rockville". 

The following week was" spent training interviewers in 
the field. The week of January 26, 1981, 109 interviewers were 
trained in two sites-Cherry Hill, N.J., and San Antonio ,- Texas . 
The week of February 2, 1981*, a total;of 94 interviewers were - 
trained in S.t. Louis, Missouri, "and San Diego, California. Each 
session was attended by representatives from ED and Advanced 
Technology, the Westat project director, two lead trainers, 
aupervisorsT^'d-^Ker tSIHeTs" ratio~~wa"s 
"never more than 1 trainer for each 10 trainees. 

Interviewer training for the BEOG survey was based on a 
training plan which Westat has developed and refined for other 



large-scale surveys . Training techniques included' -home study of 
a programmed text, interactive lectures to the large group/ and * 
Vrole-playing (administration of a mock interview complete with 
documentation) . Central to the training plan is the subdivision 
of the^?*oup of trainees into "learning communities" of approxi- 
mately^O trainees who 'receive individual attention from the 

"community leader." For the BEOG survey, Field Supervisors 

r ... 
functioned in the role of community leaders during training so 

they were able to ^observe and assist the interviewers they later 
supervised. , Additional trainers also acted as community leaders 
and reported the progress of each trainee to their supervisors at 
the. end of each day of training. Trainees who successfully 
completed&*a practice interview and the final exam and who received 
^positive evaluations, from the comunity leader were given assign- 
ments at training. 

Field Issues • m9 

Traveling - Interviewers 

Although student respondents were generally located near the 

sampled institution, parent respondents were scattered throughout 

t & 

the entire country. This meant that at some point, some inter- 

viewers would need to travel to the respondents located in far 
away places. Assignments of this type were held and allowed to 
a r cumulate until the last few weeks of the field period. At that 



point, supervisors coordinated interviewer travel plans with the. 
Field Director, and the interviewers were sent out to t Conduct the 
interviews. 
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Interviewer Attrition 

Interviewer attrition on the BEOG study is notable in that 
it wasnevera significant problem. Generally, interviewers left 
the study only after most or all available work was completed in 
their area of* the country. 

Interviewer Production 

The chart below indicates weekly* interviewer production 
of comple'ted interviews. These totals are taken from Automated 
Survey Control System reports and represent the number of ques- 
tionnaires received each week by the Regional Supervisors. 



Week # 


1 


2 


3 


4 


■ 5 


, 6 


7 


8 


9 


10 


Completes 
This Wfeek 


212 


638 


1247 


1126 


1206 


1202 


1X59 


812 


412 


141 


Cumulative 
# Complete-s 


• 212 


850 


2097 


3223 


4429. 


5631 


6790 


7602 


8014 


8155 



Reporting Procedures and the Automated Survey Control System [aSCS] ) 

At least twice -a* week the Field Director contacted the ' 
Regional Supervisor by telephone to discuss fieldwork progress 
and any problems that had arisen. One of these phone conferences 
was used to A discuss the weekly ASCS reports. An integral part of 

-f ieid^ai^gen^-nt^^ 

management system known as ASCS. The ASCS operated through small 
computer terminals located in the supervisors 1 homes and con- 
nected through telephone lines to a Computer. A similar terminal 



was located in the home ? pff ice for use by the Field Director. 
Each week the supervisors would enter information on field pro- 
gress into the system, and on a regular basis the terminal would 
print <j>ut summary ^reports on survey progress. The system was 
also used to transmit; and receive messages to and from the hdme 
officii, as well a's from othelfc; Regional Supervisors. 

The ASCS generated three ^reports which were used f by the 
Regiohal Supervisors. ASCS Report #1, the Supervisor Interviewer 

Report, listed the I.D. number of all cases currently assignee] to 

. I S 
an interviewer. Each week a new Report #1 was generated for each 

interviewer. This report served as a record of assignments and 
was discussed during the weekly supervisor/interviewer confer- 
ence. , ASCS Report #4 was also generated' weekly and presented, * 
production information on all interviewers! It provided such 

things its response rate, hours and expenses per complete, and 

\ \ f 9 * 

cost peir Completed interview. "ASCS Report #5 presented totals 

\ \\ ' . . 

showing the current disposition of all cases in a region. A 

f c ' 

ireview of chis report provided an afceurate , overall picture of a 
region's progress in completing the survey. 

In addition to these reports, the ASCS was capable of 
generating l\ more reports for use by home office staff. These 
reports were used to monitor nationwide survey progress an\i 
provided detailed' information about different respondent types 



~(e .g dependent students of dependent parents) . 
Field Problems 

No major problems arose during the fie^l per iod, and minor 

issues were discussed with the field staff .through the use of 

Interviewer F k ield Memos. * _ 
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SECONDARY DATA COLLECTIO N * 

; \ 

Secondary data collected^ for the student/parent sample 
consisted Oif the following: 

• Student Eligibility Reports [SER] - 

• IRS tax forms 

V 

Financial institution records 

i <* • 

• Tax assessor records 

Student Eligibility Reports Obtained froift Schools • 

A list of the -students selected for study participation was 
sent to each sampled school* with a request for copies of the SER 
on ifile for each of the selected students and a current mailing 
address and number. Of thS 307 schools sampled for participation 
in the fetudy, 305 schools cooperated' in sending copies of the 
SERs. A total of 4,710 SERs* were received from. all schools. 

When the SER Transcription Log for a£ .institution was 
complete, it was sent to the dalli entry office to be keypunched. 
Using the information keyed from the SER Transcription Log, 

4 

mailing labels, Call Record labels, questionnaire *£afeels, and 
mini-labels for filing were generated. In addition, a Master 
Receipt Control Log was generated to be used for receipt control 
of the other data sets to be collected during the study. 
IRS Tax Forms 

Students and parents for whom an address Was available were 
sent a letter of introduction and a package of materials to 
assist them in preparing for the interview* Included in each of 
the k student packages ^nd in the dependent parent package was a 
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list of documents they wouM^be asked to show the interviewer. 
This list was "customized" for >each. respondent, based on informa- 
tion from the SER. If the SER indicated the parent or the 
student had filed a 1979 tax return, an "IRS Form^ 4506 , 4 Request 
for Copy of Tax Form" and an instruction sheet for completing the 
request were included for the respondent^to fill^out and return 
to Westat. Finally ,y^an "information update ■ sheet" was x enclosed 
for the student or parent to fill out yXth the current names, 
addresses, and telephone numbers of thd student and parent. A 
pre-addressed- postpaid envelope labeled with the respondent's 
study identification number was enclosed in the package to be 
used for Returning the information update sheet and the Reque^J: 
f©r Copy of Tax Form. In addition, Request for Cc5py- Forms were 

obtained from respondents by interviewers at the time of the 

« 

interview. , ' 

IRS Request for Copy Forms returned through the mail were 
labeled with the study identification number written on the 
postpaid envelope. The date the forms were received was recorded 
in the Master Receipt Control Log. These forms w£re then pro- 
cessed, along with the Request for Copy Forms obtained during the 

interview, and sent to the appropriate IRS service^ center. 

t • 

IRS 1040 and 1040A form photocopies were * sent to the receipt 
control office by the IRS Service Centers* in packages containing 
an invoice', a list of the photocopies contained in the. shipment, 
a list of the requests unfilled because the service center was 
not able to locate the form, and a list. of requests unfilled 
because- the service center had no record af the requested form. 
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Financial Institution Records 

A Financial Institution Authorization to Release Information 
Form was completed in the .field for each account maintained by a. 
respondent who claimed to have more than $4,000 in checking and 
savings accounts at the time of the BEOG application. The com- 
pleted authorization forms were removed from the questionnaire 
booklet at the time the case was processed through receipt 
control.- 

Two copies were made of each release form. The original 
release and one copy were sent to the manager of the financial 
institution named by. the respondent, along with a covering 
letter explaining the study and the participation required from 
the bank manager. Each financial institution was asked to ascer- 
tain the respondent's account balance as of the date of Applica- 
tion for a Basic grant and to record the balance on the release 
form. The release form was then tp be returned to the receipt 
control office in a postpaid envelope. 

A' total of 422 completed Authorizations to Release Informa- 
ti,on were obtained from 302 respondents reporting more than $3,999 
in checking and savings accounts and sent to financial institution 
managers. The managers were requested to report the respondent's 
'account balance on- the date pf application. The 375 completed 
returned forms provided financial information on 270 respondents^ 
Tax Assessor Record Study Results 

A total of 78 schools, or 25 percent of the schools in the ' 
sample, were systematically selected after being stratified by 



size and type for inclusion in a study of tax assessors records 
of hone value. All homeowners in the 78 schools were selected 
for study • • Of the^l,260 sampled applicat ions\in the 78 schools, 
568, or 45.1 percent, were dependent or independent student 
homeowners . 1 

Questionnaires requesting reports of the most recently 
assessed market value were sent to local tax assessors based on 
the current address recorded on the Student Eligibility Report. 
The assessors were to return the completed questionnaires to the 
receipt control office in ^the postpaid envelope provided. Tele- 
phone follow-up was used to prompt late responders. A to^al of 
466 assessors responded with usable data. - * 



INSTITUTIONAL DATA COLLECTION 

This section describes all aspects of the institutional data 

f * 

collection , including : 

Development and field testing of the data collection 
instruments 

Site visit scheduling 

Interviewer recruitment and training 

Field procedure * ' 

Quality control procedures and field supervision 

i 

Post 'data collection debriefing 
The data collection instruments used in the institutional 

component of l^jhe study were developed duri'ng the fir^t weeks of 

4 ... 
the study froij the instruments used in the previous BEOG Quality 

Control study!! Buildin'g, upon the experience of the earlier 



study, the Institutional Interview Questionnaire r IQ] and the 
^Student Record Abstract [SRA] were revised and refined. Once the 
final versions of the instruments wer;e .produced , they were sub- ^ 
nitted for Federal forms clearance. ~* 

A series of field tests played an integral part in the 
development/-o'f the instrument^ . The results of the field tests 
were used to (H revise and improve the instruments, (2) estab- 
lish field data collection procedures, and (3) develop inter- 
viewer training materials.. 

Nine institutions located in the Washington metropolitan 
area and roughly representative of the larger sample we're con- 
tacted to participate in the field tests. Figure 3-3 shows these 
institutions and tHeir characteristics. 

The field tests were conducted by sehior project ?<taff in 
two-person teams. fne member conducted the interview and the 
second recorded commerr&a regarding responses to the items, diffi- 
culty in understanding the questions, the order? of the questions, 
questions demanding further probing by the interviewer, or any 

other observations -that might have helped in the revision at the 

■ - > * ' 

instrument . Following the interview, the FAA was asked to cri- 
tique both the questions and the interviewing techniques used. 

. The financial aid ( files r of n ^^- Basic Grant . recipients • 
selected at' random by^the FAA wer Previewed, and their informa- 
tion was recorded in the' SRAs. The Bursar and 'Registrar were 
visited if financial or registration informatiqji w^s* not 
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Princ^&^b*ig^^^mrnunity College 
Northern^irgjb^^'Comnrunity College 
Montgomery Co^efje: 
Catholic Una£eHMy : * ' ' v rt ' 
George WasTii&'g^cpil 'Ifc^yersity 



CHARACTERISTICS 



HoWarS Univef^^y 

«. * 

George Mason University 
Strayer Business College 



Control Data. .Iristitute 



2r-y'ear/public 

2-year/public m 

2-year/public 

4 -year /private 

'4-year /private 

4-y ear/private 

4-year /public 

Less than 2- 
year /proprie- 
tary 

Less than 2- 

year/proprie- 

tary 



, m v 



FIGURE 3-3 

INSTITUTIONS PARTICIPATING IN PILOT 
TESTING OF INSTITUTIONAL , DATA COLLECTION INSTRUMENTS 
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•/ available in the financial aid office. While at the institu- 
tions, detailed notes were taken on the effectiveness of the • 
Student -Record Abstract, the length of tine of each file 3- 

c 

review, and any "problems that night be encountered in locating ^ 
student .financial aid data.* ' 
Site Visit Scheduling 

i 

. In designing the field organization and travel itineraries, 
every effort was nade to balance the need* to minimize travel 
expenses with the, need to allow . sufficient tine for data collec- 
tion, travel, and rest. After contacting and gaining compliance * 
from all v 305 institutions, a firri site visit schedule—including 
airline, hotel, and car rental arrangenents — was established. 
Interviewers were required to adhere to this reestablished itin- 
erary as closely as possible. Each interviewer's f ield^schedule 
was monitored clos.ely by the project office throughout the eight 
weeks of data collection. A step-by^step description of the 
procedures used to establi sh ' a site "visit schedule follows. 
s 1. After field, testing-<Ua^ survey instrunents and the field 

procedure, we estimated that an interviewed could complete 12 
£ile reviews a day. Based 'on this knowledge of what constituted 

an average work day, the number of work days for each of the 305 

* 

institutions in the sample was calculated. ^ 

' r \ ° f COntinental United States was divided into 

13 cj^J^^^X re 9 ions / each with approximately the same number of 
^orki^ys^ travel days, and rest days. Every effort, was made to 
£" capture "clusters" of institutions within each reqion. The 
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average region contained 24 institutions (mote where institutions 
were relatively dense, such as the mid-AtJantic states, fewer 
where institions were relatively sparse , such as the Rocky 
Mountain states). " 

3. A tentative site visdt schedule for each region was 
established with travel routes within each region designed to 
minimize travel expenses. 

4. Advanced Technology contacted ea'ch of the 305 institu- 
tions at least 3 times prior to the site visits. In January 
1981, two months before the data collection began, letters were' 
sent to all institutions describing the study's objectives and 
asking for participation. Telephone calls followed two weeks 
later.* During these calls, senior project s^aff followed a 
defined protocol. They: 

• Asked if the FAA had received and read the initial let- 
• , *ter 

m Reviewed the purpose and requirements of the data 
collection effort ^ 

• Informed the FAA of the tentative site visit dates 

• Asked if the dates were acceptable 

• Asked for acceptable alternate dates if the initial 
site visit dates were unacceptable 

• Asked for any specific information that wpuld simplify 
the interviewer's task (i.e., directions to the aid 

0 office, parking arrangements, and location of the 
Bursar and Registrar) 

5. Following the telephone calls, a firm site visit . . >^ 
t schedule was established and letters were sent to a3J. institu- 
tions confirming appointment times • 
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Interviewer Recruitment and Training 

+ Advanced Technology advertised for interviewers in najor 

city newspapers, the Chronicle of Higher Education , and the 

NAgFAA Newsletter . Nearly 100 resunes were received and 

reviewed; 3^ individuals were interviewed; and 13 were hired. 

All 13 had considerable financial aid experience; all had 

bachelor r s , degrees-; an<5 3 had advanced degrees. We felt that 

* y 
interviewers with student aid experience would be far more 

effective data collectors than those without such experience. 
Experienced student aid administrators would be able to effec- 
tively probe for answers from FAAs; they would b<* able to recog- 
nize significant answers;, they would know when inpressive 
sounfling jargon was actually vaque or meaningless; and they would 
collect data on. individual students quickly and efficiently given 
their familiarity with the record keeping practices of institu- * 
tions of higher education. 

Two manuals were developed rfor training . The Interviewer 
Training Manual describes the study and outlines precisely the 

procedures to be followed at each site, including information on: 

i 

„ a 

• Recording student record data ,r 

• * Editing 

• Returning forms to the project oT^ice 

• Travel lqgi sties 

The Qiiestion-by-Question Specifications Manual describes 
each question in the interview and record review form, giving 
information on the various .responses to expect. 
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The interviewers were trained the vfeek of March 23, 1981, in 

*■ 

McLean, Virginia, in all the data collection activities they 
subsequently pursued in their field* work,. In addition,- they 
received' training^related- to reporting,* scheduling, expense 
reports, and mailing. and receiving survey forms- The training 
included field practice at seven Washington, D,C, metropolitan 
area institutions. " " 

A day-by-day description of the principal training activi- 

ties follows: w * / ( 

Monday morning Review of the background of the BEOG Quality 

Control • study and Campus-based programs, and 
summary of the interviewers' tasks and 

* ' responsibilities 

Introduction to ■ Institutional* Interview Form; 
presentation of 'film: Introduction to Inter - 
viewing ; demonstration of institutional 
interview ^ 

Role playing of institutional interview with 
trainees in pair s ; " iten^by-item exam- 
ination of Student Record Abstract [SRA] 

Completion of four SRAs using hypothetical 
student files of varying complexity; explan- 
ation of Alternate Disbursement System - . 

> ' . < l 

Role playing of resolution/exit interview 
with trainees in pairs; explanation and 
practices .with the Corrections Control Group 
forme 

Explanation of shipping and receiving ' of' data 
collection materials, field editing metho- 
dology, and travel arrangements; explanation 
of *data. receipt and processing system 



afternoon 



Tuesday .morning 



. afternoon 



Wednesday morning 



afternoon 
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Thursday Field practice at seven metropolitan 
Vfashington institutions 

.Friday Detailed review of field practice; completion 
of paperwork 

Field Procedures 

This section provides a summary of the procedures used to 
conduct the institutional interview and to administer the survey 
instruments. (For a more detailed discussion of the interviewing 
.protocol and the procedures used to complete,, the SRAs and CCGs, 
refer toAdvanced Technology's Interview Training Manual ). 
Figure 3-4 illustrates the complete data collection cycle discus- 
sed in this section, from the confirmation of the interview time 
to the, sending of completed forms to the project office. A 
step-by-step explanation of the procedures in this cycle follows: 

1. At least two days — but not mOre than 'bne week — prior to 
the site ^sit, the interviewers called the FAA to confirm their 
appointment time and to ^t instructions for parking and locating 
the financial aid office . 

-2. "After arriving at the institution, and before adminis- 
tering the- interview, the interviewers met with the FAA to intro- 
duce themselves and the study. The following are items that were 
covered in the introduction: 

• A description -of the purpose and nature of the fetudy, 
witti particular emphasis on the goals of the institu- 
tional component of the study. Even though they had 
received letters and phone calls, many FAAs had only a 
sketchy idea of the study pfiorSiO the site visits. ^ 
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1. PREPARE FOR 
INTERVIEW* CHECK: 

• MATERIALS 

• APPT. TIME 

■ DIRECTIONS/ 
ROUTE / 



2a. INTRODUCE: 

• YOURSELF 

• QC STUDY 

DISCUSS CONFIDENTIALITY 
PROCEDURES 



# 2b. INTERVIEW 
' FAO 



2c. SCHEDULE 

EXIT INTERVIEW 



3a. COMPLETE 
ABSTRACTS 
AND AWARD 
CALCULATIONS 



3b. VISIT 

REGISTRAR 



3c. VISIT 
BURSAR 



4. COMPLETE 
CORRECTIONS 
CONTROL GROUP 
FOR*MS 




NO 



5a. RESOLVE WITH 

• FAO 

• BURSAR 

• REGISTRAR 



5b. FAO COURTESY 
CALL/EXIT 
INTERVIEW 



6. FIELD 
EDIT 



7. COMPLETE 
TRANSMITTAL 
FORM 



8. BUNDLE DATA 
AND INSERT INTO 
PRE ADDRESSED 
MAILER 



9. SEAL TAPE. 
AND MAIL DATA 
PACKAGE 



10. RECORD 

MAILING DATE 
AND LOCATION 
IN NOTEBOOK 



11. CALL SUPERVISOR, 
BE READY TO 

DISCUSS: 

• PRODUCTION TO- 

• DATE 

• EXPENSE REPORT 

• ANY QUESTIONS OR 
PROBLEMS 



1 
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FIGURE 3-4 
INSTITUTIONAL DATA .COLLECTION CYCLE 



• An assurance of the confidential ' nature Q.f the study. 
The interviewers read the "Confidentiality Statement " 
found on the cover of the IQ and, if it seemed that the 
FAA was particularly sensitive about the confidential- 
ity issue, explained the various measures employed dur- N 
ing the data collection and data processing stages of 
the study to maintain confidentiality. 

• A brief step-by-step itinerary of the data collection ' 
activities at the institution, describing the general 
purpose of each step. 

3. The interview, normally lasting from 60 to 90 minutes, 
followed. To assure complete, accurate, and consistent data, the 
interviewers were instructed to observe the following basic 
guidelines when asking questions and recording responses: 

Remain neutral * 

> 

Ask all questions exactly as worded 
Discourage unrelated conversation 

Ask respondents to enlarge or clarify answers when nec- 
essary 

Record verbatim the FAAs ' responses- 
At the conclusion of the interview, a follow-up or exit 
interview was scheduled with the FAA. 

4. After the initial interview with the * FAA, the inter- 
viewers reviewed student files and recorded the information in 
the SRA. A notice stating the purpose and date of the data 
collection was placed in each reviewed file. 

* 5. Before leaving a site, the interviewers conducted a 
brief exit interview with the FAA. The purpose of the exit 
interview was to thank the FAA for his or *her cooperation and to 
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discuss SRA discrepancies to learn whether the FAA could offer a 
logical explanation for what on the surface appeared to be an . 
error or .violation of the BfeOG program. The interviewers were 
instructed to use tact in seeking these explanations, reassuring 
the FAAs, if necessary, that their responses would be kept in 
confidence and used for national estimates otil^j. 

6. At the end of each workday, all completed instruments 
were reviewed for possible omissions, inconsistencies, illegibly* 

,c 

handwriting, or misplaced codes. If interviewers were scheduled 
to return to the institution the following day they could take 
advantage of this opportunity to clarify or retrieve any missing 
information.* * Otherwise, interviewers did not recontact institu- 
tions for data retrieval purposes. 

7. The interviewers were instructed to mail survey data to 

a 

the project office every two or three workdays or before a week- 
end or extended travel v period. The interviewers recorded on a 
transmittal form the name of the institution ( s ) , the number of 
specific items being returned in the maimer, and all other infor- 
mation that explained the status of the returned data. The*trans- 
mittal form and the survey forms were secured with a rubber band 
and placed in a pre-addressed , business., reply mailer. "Upon 
receipt at the project office, the contents of the mailed pack- 
ages were verified against in-house records of interviewer 
assignments. (See Chapter 4 f or ,a further discussion of receipt 
procedures . ) 
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Data Collection Supervision and Quality Control 

In order to properly conduct a field data collection of this 
magnitude, it was necetsary to establish well-defined procedures 
to ensure: . / 

• The quality of survey data ; 

• The confidentiality of survey data 

c 

• Full communication between field and project office 
staff 

• The orderly flow of survey materials between the * 
project office and the field interviewers 

Quality -Control * . 

Rigorous quality control procedures were required to ensure 
the validity and reliability of the collected data. These 
included project office preview of the dat£, project office tele- 
phone validation, and on-site observation of interviewers by pro- 
ject staff. 

< 

1. Project Office Edits t X 

After the instruments arrived in the project office they 
were scrutinized by a series of manual and computerized edits. 
When critical omissions or ambiguities' were discovered in the 
questionnaires, the interviewer 'was contacted" immediately for an 
explanation. OccasionW^y an institution was contacted directly 
if survey data needed clarifying. A detailed description of the 
project office's coding and editing system can be -found in 
Chapter 4. * 
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2 . Project Office Validation 

Project office validation verified that the institutional 
data collection was, in fact, conducted according to correct pro- 
cedures. Each week all institutions reported as having been 
visited the previous week were telephoned to confirm that the FAA 
was interviewed, that the student records wer^ inspected, and 
that the conduct of the interviewer was appropriate. In addition 
to this general performance validation, one institution was 
selected fron among those that each interviewer had visited that 
week. That institution was asked to verify two items on the IQ 
and one item on a randomly selected SRA. The items selected for 
validation were those unlikely to have changed in fact or in the 
respondent's perception between the time of the site visit and 
the validation call. The findings of each call were carefully 
documented. Institutions where data collection was observed by a 
home office staff member were not called for data validation. If 
an FAA had made a negative evaluation or a discrepancy had been 
discovered between the interviewer's findings and the validator 1 s 
findings, the interviewer would have been contacted immediately 
for an explanation. Virtually all the FAAs called, however, gave 
very positive evaluations of the interviewers 1 performance during 
the site^visits. In addition, no data discrepancies were found." 

3 . Field Observations 

Field visits allowed the project staff to observe first hand 
the data collectors 1 interviewing techniques, professional manner 



and thoroughness. A report was written for each site visit, 
identifying areas of weakness and noncompliance with approved 
procedures. Soon 'after the interview and file reviews—and while 
still on-site — the field monitor reviewed each item on the report 
with the interviewer, noting" strengths and weaknesses. If areas 
of improvement were identified, the' field monitor added specific 
suggestions for improvement -on the field report. Each inter- 
viewer was observed twice: once during the second week, of the 
data collection and once during the sixth week. After each site 
visit, the fiel^d monitors r\et to discuss their ' findings . If ger^ 
eral areas of improvement were identified, they were noted' in the 
next memorandum sent to all field interviewees . ■ V 
Confidentiality Procedures * 

^ To protect the confidentiality of the survey data, the fol- 
lowing procedures were employed during the data collection: 

• This statement (found' on the front of every IQ) was 
read to FAAs before each interview. 

0 CONFIDENTIALITY AND ..PRIVACY 

This study is being conducted according to the regula- 
tions and provisions of Subsection (e) (3) of the Pri- 
vacy Act of 1974, as amended. The information (i/we) 
collect at your institution will become part of the 
existing BEOG system of records; however, that data* 
will be aggregated in such a .way as to make identifi- 
cation of, a particular institution's records impos- 
sible, (i/we) have signed a confidentiality statement 
and, except for the express purpose of this study, 
(i/we) have sworn not to reveal any information you 
. give (me/us) during this interview or from (my/our) 
review qf your student files, except as required by 
' law. 
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• All interviewers signed an assurance of confidentiality 
statenent . 

* * 

• All interviewers kept completely confidential the names 
of respondents, all information or opinions collected 
in the course of interviews, and any information about 
respondents learned incidentally. ♦ 

m Survey data containing personal identifiers were kept 
in a locked container or a locked room when not being 
used each working day *in routine survey activities. 

^ -Communication and 'Reporting 

Full communication between field and project staff was cru-- 

cial to ensuring qpality data and maintaining the tight field 

schedule balled for in the survey. Supervision of the data col- 

lection effort took place primarily through scheduled weekly 

telephone calls from the field staff to the project office. 

Periodic memoranda served as a means for informing field stafrf of 

any update to or changes in data collection procedures. 

*■ 

1 . Telephone Procedure 1 

A separate telephone line with an 800 number and a recording 

Y 

device' was installed in the pro3ect office. Each week*on a 
scheduled day.^and time interviewers were .required tp call the * 
project office. These weekly calls served three purposes: 

• : Monitoring of Data Collection 

The calls provided an opportunity for thte data collec- 
tion manager t$ review with the interviewer any prob- 
lems or error 'patterns identified l5^the coding and 
editing staff. (See Chapter 4 for farther discussion 
% \ of' this procedure.) The da£a collection nahager dur- 
ing this tine'answered questions regarding interview- 
ing procedure, survey form administration, and coding 
convention. ^These weekly calls also gav£' the data col- 
lection manager the ' opportunity to discujss the contents 
of the field -memoranda with the interviewers. 

- i • 
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- • granges. a-nUI nterview, "-Travel , and Accommodation' 
:&rrang events J > ^ 

-V/.Bv^zy* effort was made ptior.to the field data colleo- 
// tloit ■ : to • establish a. schedule -thatVwould ^allow suffici- 
-V-verit -time for interviewing, - file reviews travel, rest/ 
-~Vrict ijieals* Often-,. -howjev££, th^re were unforeseen^ 
"\ changes in the interviewers 1 itinerary initiated by " 
- ** * '-PABSav hotjels, airlines / or, the interviewers themselves. 
L .Jylti all" oarsfcs the interviewers; were required c to report 

\ schedule changes to th£ project office. Since the 

• , ltiixsrajeA^s rwere arranged centrally, at the project 
i^/o^Tibe,, the data -collection manager was frequently 

" ' inforfne'd about. fraVe.l -a rrancfement^ changes prior to the 
< -field staff. in these-*£i&ses , . the data, collection mana- 

ger informed the interviewers .about schedule changes. - 
If the progress of a particitfLar interviewer was found 
/ to lag due to unforeseen datfe collection or travel 

problems, the data collection manager discussed with 
the interviewers ways of tesolving the schedule' prob- 
lems during £he -weekly calls. Sites of interviewers 
m whose progress was lagging were often -reassigned to 
other interviewers who were ahead of schedule. In 
emergencies, project office staff were sfcnt to the- 
field to aid interviewers with schedule problems. 

• Clarification of .Routine Business Matters 

Issues related to expense reports, travel advances, 
paychecks, and mailing procedures were also discussed 
^ during the weekly calls. 

~* The interviewers were encouraged to call the project office 

more frequently than the required weekly call. . A recording* 

device was installed to take messages after business hours, and 

all interviewers were given the telephone number of a staff mem- - 

ber to call in case of an emergency. 

2. Field Memoranda 

— r^r 

Field memoranda wfere issued to communicate updates to exist- 
^>sing procedures or implementation of new procedures. To ensure 
that the interviewers understood each item of the memos, the data 
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collection manager discussed the memos with the interviewers 
during the weekly calls. 
Survey Materials Management 

A mailing'^ystem was established to ensure *that the inter- 
viewers were adequately supplied with survey materials during the 
eight weeks of data collection. Materials were nailed to field 
personnel on a periodic basis' throughout ^he data collection 

4 ** 

effort . Well in advance of site visits a separate mailing 

schedule was established for each of the interviewers. These 
* . 

nailing schedules were used to determine the number of institu- 
tions each package would cover, the exact number^>f all instru- 
ments required for these institutions, and the institution most 
appropriate' to receive a^rid^ hold the package for the interviewer. 
The packages were mailed three weeks prior to the scheduled 

site visit t One week prior to the schedule^ site visit the 

. — * 

* ■» * 

institution was called to verify the, arrival of the package. If 
the package )jad not arrived, a duplicate package was prepared and 

sent by express mail. * 

Interviewer Debriefing 

'The debriefing* of , the institutional interviewers, held on 
May 23, 1981, was- an integral part of the overall data collection 
effort. In attendance were \ the 13 interviewers, all the home 
project staff, and the Project O/ficer. Each of the, interviewers 
hacl considerable student financial aid experience and, therefore, 
had much to offer during the day-long discussion in the way of 



observations and recommendations regarding the Basic Grant 
^ 

delivery system. 

v * 

V. 

The debriefing was organized into five sessions designed to 
focus the discussion and assure that all pertinent issues were 
addressed. A summary of the principal debriefing topics follows 

• BEOG Delivery at » the Institutional Level " 

1 — " ; 

The interviewers compared institutional procedures 
observed during the data qpliection, discussed institu 
tional validation, recalled significant problems with 
Basic Grant delivery noted at institutions^ and made 
recommendations for improvement to the Basic Grant 
program. 

V 

• Evaluation o f Dat a Collection Instruments 
■ ~t 

The interviewers singled out questions that^ needed 
~ improvement . 

• Evaluation of Training Program 

The interviewers discussed how well the training had 
prepared them for their field experiences and noted 
areas where instruction or greater emphasis was needed 

V 

• Status Report on Analysis of Institutional Data 

A senior staff member gave a ihort overview of the* 
major findings to date. 

• , Evaluation of- Logistical Support 

The interviewers evaluated the itineraries, telephonin< 
procedure, field memoranda, field observation, hotel 
and travel arrangements, cash handling procedures, and 
general staff support. 
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CHAPTER 4 
DATA PROCESSING 

4 

r 

\ 

The first section of this chapter describes the procedures 
used by Westat tt). compile, edit, and enter student', parent and 
secondary data. The second section discusses the procedures used 
by, Advanced Technology to process institutional data. The final 
section provides a technical discussion of the final data file 
merge. 

■ 

STUDENT/PARENT 

7 ' 

"Six" primary ~^al^ 
component of the BEOG Quality Control study. The preparation of 

each data set required the application of similar data prepara- 

\ 

tion procedure^. These procedures are described in this section 
of the report. ^ y 

Receipt Control 

The function of receipt control was to provide a* catalogue 
of all data documents received by the Coding Office of the Basic 
Grant study. The receipt control procedure was. slightly different 
for each of the data sets. 

As >SERs were received from the sample schools they were 
logged in on a cojnputei^ printed listing of the students sampled 
from the schools. Data from the SERs and data provided by the 
schools yere used to produce a label file of names, addresses, 
and Social Security numbers of all sampled students, and the 
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names and addresses of their parents. This label file was used 
to produce a master receipt control log for cataloguing Student 
and Parent Questionnaires and IRS forms returned from the field. 

The master receipt control lotf listed the study identi- 
fication number, name, address, and Social Security number of 
eabh student/parent pair. The log was organized in numerical 
9rder by study identifier within school. Space was provided for 
recording updated infqrmation on names and addresses, the date 
each questionnaire was ^received from the field, the completion 
status of the document, and the coding batch number assigned to 
the document^ Lab^I¥3~caIuit^ "Tecor3Eng 
the receipt of IRS Release Forms by mail (from the initial infor- 
mation mailing) and inside the questionnaires (obtained during 
the interview) . When IRS 1040 and 1040A Forms were deceived from 
the IRS Service Centers, they were coded with case identifiers 
and logged in tfte master log, with the IRS coding batch number. 
Coding ^ 

A coding manual was prepared for the Basic Grant study for _ 
use* in training the data preparation staff; serying as a complete 
and detailed reference for analysts, programmers, and data 
preparation staff; and providing documentation for the Basic 
Grant study data files. 0 The coding manua^ ^consists of an intro- , 
duction^ to the study procedures and purposes, a review of general 
data preparation procedures to be followed, and coding and 
editing specifications for each of the six data sets. 



Approximately 40 survey processing personnel were selected 
for training as coding and editing staff for the Basic Grant 
* study. Four group leaders were selected from this number based 
on their skills and qualifications. The group leaders were 
assigned as assistants to the coding, supervisors and as coder 
verifiers. As much as possible, experienced Westat survey 
processing personnel were selected as coders 1 for this project in 
order to. minimize the amount of training necessary on basic 
coding skills. ^ f * 

Coders were- trained' in groups of between 4 and 14. Each \ 
group was trained to code on one 'of the six study data "sets. x 
Between one and eight hours* of? training time were required, 
depending on which data 'set was to be coded. 

Questi6nnaires , were precolumned before printing, so that 
coding could be written directly orx the questionnaires. SERs and 
IRS 1040 and 1040A Form's could not be precolumned, so transcription 
sheets werfe designed for the cpding. : 

Coders! were assigned work^by ceding batch and were required 
to complete \the coding of one batch before beginning work on 
another. Errors foynd during verificatipn by the sup^visor were 
discussed with* the cod^r .committing them. ' If. persistent errors 
were discovered, a cocier >would be asked to review previous batches 
and Jsordfecrt tnem. Problems found duriiig coding but ".not resolved 
in thef cociing Specifications were documented aild referred to a 
supervisor to b^e resolved. 



Occasional problems with illegible figures arose in the 
coding of 'photocopies of Student Eligibility* Reports and IRS tax 
forms It was necessary to code illegible data elements as 
missing values in these situations* 

The major coding problem for the Student and Parent Question- 
naires was the large number of questions in each questionnaire 
which were open-ended in format. It *was not possible prior to 
the beginning of coding to devise lists of all the possible 
responses to these items. Since this problem was expected, a 
controlled system of dealing^ with it was implemented at the 
beginning of coding: Responses which were not codable in the 
predetermined list of codes in the coding specifications were 
documented and referred to the supervisors who constructed codes 
for the new items. New codes were published each morning on 
a Coding Change Sheet. 'Coders were responsible for keeping their 

manuals up to date and were required to record each issue df the 

t 

coding changes in a log. 
Data Retrieval 

Coders were trained to "edit" the data collection instrument- 
during the coding. The edit function' involve^ checking for 
readability, sensibility, and following of skip patterns. (The 
editing function was much more important x in the coding of the 
questionnaires than in the coding of the secondary data sources.) 
A general rule was established that all primary verification 
questions in the questionnaire must have codable responses. When 
coders ; found erroneous skips, illegible answers, or illogical 



responses in any of the verification questions, they documented 
the problem and referred the case to a supervisor for data 
retrieval. 

Experienced, specially trained telephone interviewers were 
used fojTtelephone data retrieval of the problem items, 'case 
problems were described on a Data Retrieval Request Form which 
also served as a Record of Calls for the interviewer. Data 
retrieval was attempted on 201 Parent Questionnaires and 139 
Student Questionnaires. 

In addition to data retrieval requests due to problems found 
in the coding edit, data retrieval requests were also generated 
' during^nachine editing. In fact, a substantial portion of the 
data retrieval requested on the Parent Questionnaires occurred « 
during an attempt to obtain more reasonable estimates of the 
v^lue of assets and debts on assets^from respondents. 
Key Data Entry "and Machine Editing 

Co5ed documents and questionnaires which had been verified 
were transmitted to the Westat data entry staff in groups (called 
"keying batches") of, approximately 100 documents. . Coded documents 
were keyed into an in-house disk storage system, and then key 
verified from the disk. After keying and key verifying, the data 
were transmitted via telephone link to the main computer where 
they were stored' on tapes to await machine editing. 

All data sets except the Tax Assessor File and the Financial 
Institutions File were machine edited with special purpose COBOL 
, , programs, ^written to check for out-of-range codes, incorrect skip 
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patterns, and inconsistent response patterns. The data sets were 
edited by keying batch. 

Machine edit staff were trained coders, the majority of whom 
had previous Westat experience as machine edit clerks. A super- 
visor with machine editing experience was assigned to oversee the 
machine editing of the questionnaire files. . 

File updating instructions were written on transcription 
sheets by the machine edit clerks, checked by the machine edit 
supervisor, and then sent to the data entry office for keying and 
transmittal to the computer center. Updates were made to the 

files by a special purpose COBOL update program. After each 

_ * > *• 

update run was complete, another editing cycle- was run to verify 
that corrections had been made arid to check for new errors. The 
update-edit cycle was repeated until' each batch of data in the 
.data set was clean. 

When data sets were complete and cleaned in the machine edit 
process, a cross-fiie merge to check for missing cases and 
inconsistent student status (dependent/independent) matches was v 
performed. Mismatches between internal status (based on answers 
to questions in the questionnaire) and external status (based on 
information from the school provided copy of the Student Eligi- 
bility Report) w^re carefully checked for accuracy. Mismatches 
between the internal status of the Student Questionnaire 5nd the 
internal status of the Parent Questionnaire were also checked. 

Where true mismatches of status occurred, the original 
status of the questionnaires was altered to indicate the final 
dependency status of the case. 



In addition to the merge to compare status codes , a com- 
parison of the names of dependents listed on the parents 1 income 
tax forms to the names of the students 'was made. A^data set con- 
sisting of the study identification number and an indicator ^of 

■* • 

the search result was produced. 

Frequency distributions were run on all variables in each 
data set after the machine edit process was complete. The 
frequency distribution was proofed for any inconsistencies and 
errors not found in the machine edit process. 

» 

INSTITUTION 



This section describes the procedures used to process and 
verify institutional survey data from their receipt at Advanced 
Technology to final. entry into *a- computer data base. The survey 



form processing system^ .described in this section included quality 
control procedures designed to: 

Maintain the' confidentiality of all survey data 

• Ensure the accuracy of data provided by the 
institutional field interviewers 

© 

• , Ensure the transcription accuracy of coding clerks and 
* . keypunchers 

•* Maintain control of the status of all data collection 
instruments in order to minimize the possibility that 
[ instruments woulcT be lost 

Confidentiality of Institutional Survey Data 

Several procedures were instituted to assure the confiden- 

tiality of survey data. All incoming packages were ^p^rtalned in 

locked cabinets in a locked storage area. Access to the survey 

data was limited to those persons working on the BEOG Quality 
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Control study and to those who had been instructed in the study 1 s 
confidentiality requirements. As with the field interviewers , 
all Advanced Technology , data entry personnel were' required to 
sign confidentiality pledges. 
Receipt Control * 

The' successful completion. .of the project , given the large 
number of data collection instruments, required a well-defined 
system to be used by clerks in the receipt, logging, and routing 
of all institutional data received from the field interviewers. 
The followinq procedures were .used to maintain control of the 
,status of all cases still out in the field as well as the loca- 
tion of every form received in the home office: 

1. All incoming packages of instruments contained a r 
transmittal form describing in detail the contents of each pack- 
age. If a discrepancy yas noted between the actual contents of 
the package and its corresponding transmittal form, the inter- 
viewer was contacted immediately for 'an explanation. t 

2. The contents of each package were checked* against a 
master list of the interviewer 1 s institutional assignments. If 

it appeared that an assignment had not been completed, the inter- *} 
viewer was contacted. 

3. Once it was determined that a complete institutional 
assignment had been received at the project office, tjjie instru- ' 
ments were sorted into reference groups of manageable size. Each ♦ 
reference group, or package, was assigned an ID number which was 
recorded in a master control log. The master control log was 
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used 'to trace the path of each instrument through all the pro- 
cessing steps. It served as an excellent mechanism for maintain- 
ing control over the status and location of the survey data. 
Each 'line entry} of the log identified an instrument package by 
1 nymber and described the exact status of that package. 
Manual Editing and Coding 

A coding staff of five under the close supervision of a 
senior coder thoroughly reviewed each survey form for complete- 
ness and accuracy. - The coders checked skip patterns to see that 
they were followed correctly, checked responses for clarity and 
relevance, and checked for the consistency and logic of all data. 
Although the coders were instructed to scan all items on the sur- 
vey forms, certain questions were found to be error prone and, 
therefore, were given particular attention. 

A coding and editing specifications manual, developed by. 
senior project staff prior to the receipt of data, guided the 
coders. The manual was used to train the coding staff and served 
as a detailed reference for analysts and programmers. The manual 
included: 

• • A summary of the study 

• An explanation of general coding and editing 
procedures 

- j • Question-by-question instructions 
A list of error prone questions 
Often missing or incorrect d^ta could be reconstructed on 
the basis of responses to associated questions, interviewer 
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notes in the margins q,f the survey form, or information from 
current school catalogues and financial aid materials. 

If significant omissions or errors were discovered inter- 
viewers were contacted by the data collection manager* /In the 
rare instances when interviewers could not provide the needed 
data, the institutions were contacted directly. During the 
eight -week data collection there was never a need to revisit an 
institution to retrieve data. 

A procedure was established to record and -inform inter- 
viewers of noncritical recording errors. A log ^was used* to 
record problems a.nd suggest ions,; for improvement foi^each inter- 
viewer. At the end of each week th<* logs were routed to the 
institutional data collection manager who forwarded the infor- / 
mation to the . interviewers during their weekly telephone calls. 

Following the preliminary edit, the survey form data were • 
coded for entry into a computerized data bas^. Often a coding 
situation arose that was not directly addressed fcjy.the coding and 
editing specifications manual. If a situation could not be ade- 
quately resolved by existing coding convention or if a response 9 : 
/could not be coded with any of 'the* provided codes, the coders 
were directed to record the situation on, a form and refer itf t9 

SB 

the senior coder. The senior coder, in consultation wi*th other 

BEOG Quality Control analysts, if necessary, jna,de a decision on 

each referred case and recorded the decision ' directly onto th'e § 

"problem" form. The problem forqis were jthen f il$d by question 

number and served as a permanent recdrd-of all coding decisions. 

Often during the coding process it tfas necessary to update* the 

♦» 
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coding and editing specifications manual with jiew 'codes, changes 
in data ranges, and changes ih coding convention. It w^!T 
extremely important that all coders be informed of such additions 
and changes. As a rule, at the end of each workday, a memo was 
distributed to all coders detailing changes. The coders would 
then incorporate the changes into tl)eir manuals. 

Once completely coded and edited, instrument packages were 
routed to' the coding supervisor for verification. At the ^gin- 
ning of the coding process all survey forms were scrutinized for 
coding errors. If a pattern of errors was noted that coder was 
retrained in the deficient area. Once an individual coder 
reached an acceptable level of accuracy, a sample of that coder's 
package was inspected. In addition to the aboVe item-by-item 
verification, certain items which proved to be particularly error 
proife^were verified 100 percent in$all instruments. For example, 
the questions m the Student Record Abstract pertaifiing to insti- 
tutional error tfrere verified thoroughly. * 
Keypunching 

In the early stages -of - the project," six data entry firms 
were contacted. We decided £o engage Data X "Associates of 
Spr irigf ield, Virginia, because of its proven ability to produce 
high qualify work with speed and economy. 

Detailed keypunching instructions w$re forwarded to Data 1. 
The first keypunched packages returned. to Advarfced Technology 
were examined closely to verify that keypunch instructions were 
being adhered to. Keying was performed in packages, transferring 
data directly from 'instruments to cSrds. Each package of keyed 



data was 100 percent key verified against the associated instru- 
ments. Key verifying was performed by a key operator other than 
the one who performed the keying-. 

To maintain control over the status of all survey forms, a 
well-defined procedure was established to transmit the data to 
and from Data 1. A weekly standard time for pickup and delivery 
was set and maintained." A transmittal form detailing the con-' 
tents of each package of instruments accompanied each delivery. - 
The signature "of a Data 1 official on this form verified the 
totals of each delivery. Once returned fi:6m keypunching, each 
package was checked for completeness and filed. 
Machine Editing and Updating > 1 * 

As keypunched cards were returned from Data 1 they were 
entered on a data file and subjected to a series of computerized 
edits. The purpose of the machine edits was to (1) act as a 
backup of the manual editing, (2) perform edit£ that would be 
very time consuming if done manually, and (3)' discover key- 
punching errors. 

i 

Under the guidance of a senior analyst^ a series of EASY- 
TRIEVE m programs were designed to: 

• Check for* missing data * 

• "Check data ranges, 

• Check * for incorrect skip patterns 

m Check consistency between data items * ' 

The edit programs were tested on the first set of keypunched 
data. The results 'of these tests were reviewed b^ senior staff 
to ensure 1 that the edits had thfe proper stringency. 



Edit failures were individually fisted for resolution by the 
coding staff. The original data collection instrument was 
reviewed, and file updating instructions were written on the 
error .listing. The data file was updated and a new edit cycle, 
was run. This update-edit cycle was repeated until no data 
imperfections remained. 

Frequency distributions/ were run on each variable oncevthe 
editing process was completed The^fcstributions were inspected 
by senior staff for errors not detected by the manual or machine 
edits. If errors were found, the original instrument" was 
reviewed and- the file updated. 

SCHEDULE OF DATA MERGE 

The first step in the data merge was reformatting the clean 
rawdata tapes into SAS files. The following programs were run: 
BQCCCWLOS wa|5^pan on the SER data. 
BQCC110S was run Qji..the irs data.* " 

BQGC210S was run on the TAR (Tax Assessor Record) 
data. * 

BQCC510S was run on the FIR (Financial Institution 
Record) data. 

i 

BQGC610S was run on the parent data. 

BQC£7rOS was run on the student data. 

BQCC810S was run on the SRA (Student Record Abstract) 
data. 

t BQGC901S, was run on the institutional interview data. 
All of the SA X S programs were stored on Librarian modules. They 
were accessed and executed by short JCL sequences stored in an 
active COMNET workspace. . ^ 

. • , «-« 



SCR 

Raw Data 
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BOCCO10S 
Reformat 
Out DS010 









FIR 

I Financial Inst Record) 
Raw Data 



TAR 

(Tax Aims tor Record) 
Raw Data 



IRS 

Raw Data 



PAR 

(Parent Interview) 
Raw Data 



STU 

(Student Irtttryiaw) 
Raw Data 



SRA 

(Student Record Abstract) 
Raw Data 



BOCC510S 
Reformat 
Out DS510 




BQCC210S 
Reformat 
Out DS210 



BOCC110S 
Reformat 
Out DS110 



FWMRG 

In START DS510. PIKFIR 
Merge FIR 

Out OS122 * 



TARMRG 

In START, DS210. 
PIKIRS 
Meroa TAR 
Out. PIKTAR 



BOCC610S 
Reformat 
Out OS610 



BOCC671S 
In DS122 

Flat Parent and Student 
Data 

Out. DS671 



IRSMRG 

In- START. DS110 
Merge IRS Data 
Out PIKIRS 



8OCC710S 
Reformat 
: DS710 



O^t: 



BOCC731S 

Crosi Tabulation a on 

Student Error vj 

Problem i with Appacationi 



BOCC631S 
Croti Tabular 
Parent -Error v 
Problemi with 



BOCCB10S 
Reformat and Flag 
Student & Parent 
Response 

Out. DS810. DS321 











Institution Interview 
Raw Data 




BOCC910S 
Reformat 
Out* DS910 
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CAR 

1 Computer Applicant 

Record) 

Raw Data 
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TAPE 
Reformat 
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Student 



8QCC622S 
In OS671 
Merge Doc um anted 
Parent Oat* 

Out' OS622 



8QCC722S 
In OS722 
Merga Documented 
Student Data 
Out DS722 




6QCC822S 
In SPLIT 2 

Merge SRA Documented 
Data 

Out OS822 „ 



8QCC672S 
In SPtlT 1 

Merge Undoc Student 
and Parent Oeta 
Out OS672 



1 



8QCC631S ^ 
Croat Tabulation* on 
Perrjnt -Error v* 
Problems with Applications 



OS010 

(See 8QCCQ10S) 




8QCC672S 
In OS822 

Merge Undoc Parent 
& Student Oat* 
Out OS672 



8QCC020S 
In OS672. OS010 
Flag Discrepancies. 
Calculate SEI & Award 
Out OS020 S1 



8QCC030 

In OS811. SEI2ERO 
M*rge CAR Oat* 
Out OS030 




8QCC872S 
In OS672 

Mwgt SRA Undoc Oata 
Out OS872 
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8QCC020S 
In OS872. OS010 
Flag Discrepancies, 
CaJculat* SEI & Award 
Out OS020 S2 



MERG020 
Concatenate 
OS020 Tapes 
Out OS020. FINAL 



8QCC811S 
In OS020 

C*t cu i*t* SEI & Award 
from SRA Flag Errors 
Out OS811 SI 



i r* 



on 



BQCcfllS 
in OS020 

Caleddt* SEI & Award 
from SRA. Flag Errors 
Out 0S811 S2 



MERG811 

Concatenate OS81 1 
Tapes. 

Out OS811 FINAL 



FIGURE 4-1 
FLOWCHART OF DATA MERGE 
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The naming convention for tapes created by Data Merge pro- 
grams was BGP.BQC.DS , where the blank was filled by the 

three-digit number corresponding to the program in which it was 
created. Thus, program BQCC010S created tape BGP.BQCDS010, etc. 
The next step was to run a program called START. This program 
reads a computer applicant record tape and a tape -containing 
student, SRA, and parent data flags. These flags, created in 
- earlier rims of the datalnerge, indicate what information existed 
for each ID. The program START assigned each ID a flag to code 
which IRS information was needed, according to dependency . and 
marital status. This information was stored on a tape natmed 
*BGP. BQC. START319. 

Using BGP . BQC . START319 and the IRS data tape, the merge was 
begun by a program named IRSMRG. This "program picked out the * 
appropriate jfesr data for each ID according to the code from tape 
BG P . BQC ♦ START3 1 9 and created flag variables to indicate whether 
IRS data had been found for each item. The resulting tape was 
- PIKIRSK1. Similar** programs TARMRG and FIRMRC3 were run to collect 
tax assessor and financial institution data. Tapes created were 
' • -BGP.BQC.PIKIRSK1, BGP. BQC . PIKTARK1 , and BGP. BQC. DS122, in that 

order. The program which cataloged BGP. BQC. DS122 created' the 
."best .value" variables, to be filled through the merge. The merge 
continued with program BQCC671S, which created flags to determine 
the source, and ' documentation of the values from student and 
parent files. T^iis program read tape BGP. BQCDS122 and the 
universe file* UNIVINT3 and created tape BGP. BQC. D671. The next . 

1 • o 
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program in the sequence was BQCC622 which merged the documented 
parent data. It read tape BGP. BQC . DS671 and created } ' 
BGP. BQC .'DS622 . Then the] student data were merged by program 
BQCC722S, which read tape BGP. BQC. DS622 and created 
BGP. BQC . DS722 . At this ^oint a short program was run to delete 

the SQ * an4 PAR variables from the student and parent files. 

The* tape created still retained the best value and flag vari- 
ables, but the other variables were dropped to save space for the 
next step in the merge. 

* At this pjAnt two programs were run to Split the .observa- 
tions into two groups: those students whose dependency or 
marital status was determined by us, to have been incorrectly 
reported on the SER, and those students for whom the status items 
were correct there. The programs were named SPLITl^and' SPLIT?, * 
respectively, and created tapes named BGP. BQC .DS722 (differen- 
tiated by a fourth-level name). *This split was necessary because, 
for the first group all SER and SRA data were incorrect, while 
for the second group the SER data might still have been the best ' 
available data.. Thus, these' groups had to be split before^ 
merging the SRA data. 

A program named BQCC822S merged the 'documented SRA data into 
the SPLIT2 group. It read the second BGP. BQC . DS722 tape and 
created BGP. BQC . DS82?. The*n another short program was run to 

swap out the SRA variables and replace the SQ and PAR 

variables. This new BGP.BQC.DS872 tape contsRned the best values 
and flag variables from the latest step in the merge, as well as 
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the SQ and ^_ PAR variables. Program BQCC672S then, Merged 

the noncertifled parent and student values. One last variable 

swarp brought the SRA variables back in and dropped PAR and 

SQ variables for good. The tape created in this program was 
BGp. BQC. DS67 2. The final merge program, BpCC872S, yfs then run 



to merge undocumented SRA values. 

The SPLITl group, which contains the students with correc- 
tions to their status items, skips the SRA merging programs. 
Program BQCC672S was run on the first BGP. BQC. DS722 tape to fnerge 
in undocumented parent and stpdent data. At this point we had 
reformatted SAS data tapes of the various interview data sources 
and tapes containing the best documented values- from all the 
sources,. . Using programs BQCC631S and BQCC73lS"we compu^^c^ 
frequencies on the errors made on BEOG applications versus the 
-problems they reported with the applications. 

The two best value tapes, BGP. BQC. DS872 with the SPLIT2 
group and BGP. BQC .,DS672 with the SPLITl group were then used as 
input into program BQCC020S. This program "calculates student 
eligibility indexes [SEI] from best values and compares these' 
with the SEIs from the SER. It creates variables with the' 
discrepancy values between be^^^^^c^arxd reported items and 
variables to flag errors. The two BGP. BQC . DS020 tapes created 
were then concatenated into a single complete BGP. BQC . DSO 20. FINAL 
tape. The first two BGP. BQC. DSO 20 tapes were separately run' 
through BQCC811S which calculates awards and award ..discrepancies 
for each student. The two BGP. BQC. DS811 tapes were then 
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concatenated into a single BGP. BQC . DS811. FINAL tape, as well. 
These FllJAL tapes .were used to compute the various frequency 
tables and percentages * in Volume 2 of this report. Finally, the 
award calculation and discrepancy variables were merged with the 
computer applicant record data on tape BGP. BQC. SE I ZERO by program 
BQCC030S. 



* 



CHAPTER 5 

APPLICATION PROCESSOR DATA ENTRY ERROR ANALYSIS 

As part of the study of the BEOG application processor , an 
analysis of ejrror rate Associated with data entry was conducted. 

SAMPLE SELECTION 

The applications of approximately one-quarter of phe 4,000 

) 

BEOG recipients interviewed in the nationwide survey were selec- 
ted for the analysis., In order to maximize the use 9f time and 
resources, only applications which originated from the three Mul- 
tiple Data Entry [MDE] processors — the College Scholarship Ser- 
vice [CSS], the American- College Testing [ACT] Program, and the 
Pennsylvania Higher Education Assistance Agency [PHEAA] — were 
chosen. (The MDE processors accounted for approximately 85 per- 
cent of all BEOG initial applications during the 1980-81 proces- 
sing year.) 

The sample was drawn using Westat's BEOG Master File of all 
interviewed students and the 1980-81 Central Processor's History/ 
Correction File. Records from both files were matched on Social 
Security numbers [SSN] and the first two characters of the last 
name. From the extract of matched records, 500 CSS originated 
applications, 500 ACT originated applications/ and 250 PHEAA 
originated applications were randomly selected. <JL . 

> ERROR TABULATION 

The primary research question was "To what extent does what 
students write on their form actually become what is entered in 
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the BEOG application processing system?" ' To answer this, the 
original application forms of the sampled recipients were vis- 
ually compared with the data contained on the processor's 
History /Correction File. ^ - . 

Each of the MDE processors was* asked to provide photocopies 
of the o original applications. These photocopies were placed in 
groups of 10 and attached to a corresponding^computerized listing 
of data from the History/Correction File/ , Data were compared in 
these groups of 10. All discrepancies, /apparent discrepancies, 
or oddities between the data on the application and the data on 
the computerized listing were carefully recorded. The following 
information about these discrepant cases was logged: (1) the MDE 
processor; (2) the recipient's SSN; (3) the data item(s) in 
error; (4) the item value on the original application and the 
corresponding value on the listing; (5) the apparent nature of 
the error; and (6) whether or not the entry error had been 
corrected in subsequent transactions. As a quality control 
check, 1 completed case in each group- of 10 was reviewed by a 
supervisor. When an error that had gone unrecorded was 
discovered, all cases in the group were reviewed. 

Once all cases had been compared < senior analysts reviewed 
the log of errors. If questions arose about the nature of a par- 
ticular error, that case was retrieved and examined. The total 
number "of errors was tabulated-manually from the error log. 



